Note: This is test shard 8 of 8.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from TableRangeRebalancingTest
[ RUN ] TableRangeRebalancingTest.Basic
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240501 14:08:47.647359 6759 test_util.cc:268] Using random seed: 108247626
I20240501 14:08:47.655503 6759 ts_itest-base.cc:112] Starting cluster with:
I20240501 14:08:47.655570 6759 ts_itest-base.cc:113] --------------
I20240501 14:08:47.655583 6759 ts_itest-base.cc:114] 3 tablet servers
I20240501 14:08:47.655601 6759 ts_itest-base.cc:115] 3 replicas per TS
I20240501 14:08:47.655611 6759 ts_itest-base.cc:116] --------------
2024-05-01T14:08:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T14:08:47Z Disabled control of system clock
I20240501 14:08:47.665315 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:43167
--webserver_interface=127.6.153.254
--webserver_port=0
--builtin_ntp_servers=127.6.153.212:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.153.254:43167
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 14:08:47.770553 6771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:47.770802 6771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:47.770843 6771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:47.773953 6771 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:47.774029 6771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 14:08:47.774050 6771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:47.774067 6771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 14:08:47.774094 6771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 14:08:47.777848 6771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.6.153.254:43167
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:43167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.153.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:47.778831 6771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:47.780045 6771 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:47.787518 6776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:47.787518 6779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:47.787833 6777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:47.788237 6771 server_base.cc:1034] running on GCE node
I20240501 14:08:47.788725 6771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:47.789664 6771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:47.790827 6771 hybrid_clock.cc:648] HybridClock initialized: now 1714572527790802 us; error 42 us; skew 500 ppm
I20240501 14:08:47.793112 6771 webserver.cc:460] Webserver started at http://127.6.153.254:39479/ using document root <none> and password file <none>
I20240501 14:08:47.793678 6771 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:47.793730 6771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:47.793893 6771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:47.795586 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "8b0ff9590eb94bd78027519c194b6673"
format_stamp: "Formatted at 2024-05-01 14:08:47 on dist-test-slave-cmf0"
I20240501 14:08:47.796025 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "8b0ff9590eb94bd78027519c194b6673"
format_stamp: "Formatted at 2024-05-01 14:08:47 on dist-test-slave-cmf0"
I20240501 14:08:47.799448 6771 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240501 14:08:47.801643 6785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:47.802630 6771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:47.802765 6771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
uuid: "8b0ff9590eb94bd78027519c194b6673"
format_stamp: "Formatted at 2024-05-01 14:08:47 on dist-test-slave-cmf0"
I20240501 14:08:47.802860 6771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:47.817602 6771 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:47.818284 6771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:47.818449 6771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:47.824996 6771 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.254:43167
I20240501 14:08:47.825030 6837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.254:43167 every 8 connection(s)
I20240501 14:08:47.825980 6771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 14:08:47.828629 6838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:47.831174 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 6771
I20240501 14:08:47.831372 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 14:08:47.834682 6838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673: Bootstrap starting.
I20240501 14:08:47.837002 6838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:47.837677 6838 log.cc:824] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:47.839287 6838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673: No bootstrap required, opened a new log
I20240501 14:08:47.841745 6838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0ff9590eb94bd78027519c194b6673" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43167 } }
I20240501 14:08:47.841944 6838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:47.842005 6838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0ff9590eb94bd78027519c194b6673, State: Initialized, Role: FOLLOWER
I20240501 14:08:47.842536 6838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0ff9590eb94bd78027519c194b6673" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43167 } }
I20240501 14:08:47.842712 6838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:08:47.842782 6838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:08:47.842888 6838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:47.843706 6838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0ff9590eb94bd78027519c194b6673" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43167 } }
I20240501 14:08:47.844022 6838 leader_election.cc:302] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b0ff9590eb94bd78027519c194b6673; no voters:
I20240501 14:08:47.844318 6838 leader_election.cc:288] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:08:47.844892 6843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:47.845079 6843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [term 1 LEADER]: Becoming Leader. State: Replica: 8b0ff9590eb94bd78027519c194b6673, State: Running, Role: LEADER
I20240501 14:08:47.845382 6843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0ff9590eb94bd78027519c194b6673" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43167 } }
I20240501 14:08:47.845429 6838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 14:08:47.846725 6844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b0ff9590eb94bd78027519c194b6673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0ff9590eb94bd78027519c194b6673" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43167 } } }
I20240501 14:08:47.846837 6844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [sys.catalog]: This master's current role is: LEADER
I20240501 14:08:47.847173 6852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 14:08:47.847184 6845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b0ff9590eb94bd78027519c194b6673. Latest consensus state: current_term: 1 leader_uuid: "8b0ff9590eb94bd78027519c194b6673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0ff9590eb94bd78027519c194b6673" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43167 } } }
I20240501 14:08:47.847384 6845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673 [sys.catalog]: This master's current role is: LEADER
I20240501 14:08:47.849828 6852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 14:08:47.854987 6852 catalog_manager.cc:1347] Generated new cluster ID: d4e5138b0cbb44daa25e700ccd218875
I20240501 14:08:47.855070 6852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 14:08:47.865306 6852 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 14:08:47.866250 6852 catalog_manager.cc:1504] Loading token signing keys...
I20240501 14:08:47.877725 6852 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 8b0ff9590eb94bd78027519c194b6673: Generated new TSK 0
I20240501 14:08:47.878535 6852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 14:08:47.892623 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.193:0
--local_ip_for_outbound_sockets=127.6.153.193
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43167
--builtin_ntp_servers=127.6.153.212:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:08:47.999461 6862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:47.999661 6862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:47.999719 6862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:48.002722 6862 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:48.002794 6862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:48.002882 6862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.193
I20240501 14:08:48.006564 6862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43167
--never_fsync=true
--heap_profile_path=/tmp/kudu.6862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.193
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:48.007490 6862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:48.008563 6862 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:48.016199 6870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:48.016283 6867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:48.016574 6868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:48.017040 6862 server_base.cc:1034] running on GCE node
I20240501 14:08:48.017407 6862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:48.017920 6862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:48.019038 6862 hybrid_clock.cc:648] HybridClock initialized: now 1714572528019017 us; error 37 us; skew 500 ppm
I20240501 14:08:48.020877 6862 webserver.cc:460] Webserver started at http://127.6.153.193:39499/ using document root <none> and password file <none>
I20240501 14:08:48.021406 6862 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:48.021459 6862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:48.021646 6862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:48.023258 6862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "9c07440c3b8d4a939f56e878ef954d5e"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.023770 6862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "9c07440c3b8d4a939f56e878ef954d5e"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.026842 6862 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:08:48.028921 6876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.029891 6862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240501 14:08:48.029991 6862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "9c07440c3b8d4a939f56e878ef954d5e"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.030077 6862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:48.054231 6862 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:48.054888 6862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:48.055042 6862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:48.055615 6862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:48.056526 6862 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:48.056574 6862 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.056617 6862 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:48.056644 6862 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.064903 6862 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.193:34157
I20240501 14:08:48.064926 6989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.193:34157 every 8 connection(s)
I20240501 14:08:48.065825 6862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 14:08:48.068167 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 6862
I20240501 14:08:48.068269 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 14:08:48.070642 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.194:0
--local_ip_for_outbound_sockets=127.6.153.194
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43167
--builtin_ntp_servers=127.6.153.212:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:48.074959 6990 heartbeater.cc:347] Connected to a master server at 127.6.153.254:43167
I20240501 14:08:48.075177 6990 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:48.075711 6990 heartbeater.cc:511] Master 127.6.153.254:43167 requested a full tablet report, sending...
I20240501 14:08:48.077111 6802 ts_manager.cc:194] Registered new tserver with Master: 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157)
I20240501 14:08:48.078265 6802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.193:44041
W20240501 14:08:48.174847 6993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:48.175129 6993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:48.175197 6993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:48.178259 6993 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:48.178333 6993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:48.178426 6993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.194
I20240501 14:08:48.182011 6993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43167
--never_fsync=true
--heap_profile_path=/tmp/kudu.6993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.194
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:48.182888 6993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:48.184037 6993 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:48.191277 7001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:48.191630 6998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:48.191960 6999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:48.192315 6993 server_base.cc:1034] running on GCE node
I20240501 14:08:48.192620 6993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:48.193115 6993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:48.194262 6993 hybrid_clock.cc:648] HybridClock initialized: now 1714572528194244 us; error 33 us; skew 500 ppm
I20240501 14:08:48.195914 6993 webserver.cc:460] Webserver started at http://127.6.153.194:40219/ using document root <none> and password file <none>
I20240501 14:08:48.196409 6993 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:48.196463 6993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:48.196631 6993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:48.198190 6993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "af33d7febb294622a260df662f18821a"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.198647 6993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "af33d7febb294622a260df662f18821a"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.201803 6993 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240501 14:08:48.203863 7007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.204805 6993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 14:08:48.204905 6993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "af33d7febb294622a260df662f18821a"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.204984 6993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:48.220571 6993 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:48.221179 6993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:48.221325 6993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:48.221833 6993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:48.222697 6993 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:48.222743 6993 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.222795 6993 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:48.222821 6993 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.230708 6993 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.194:34591
I20240501 14:08:48.230738 7120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.194:34591 every 8 connection(s)
I20240501 14:08:48.231606 6993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 14:08:48.236033 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 6993
I20240501 14:08:48.236136 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 14:08:48.238258 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:0
--local_ip_for_outbound_sockets=127.6.153.195
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43167
--builtin_ntp_servers=127.6.153.212:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:48.239988 7121 heartbeater.cc:347] Connected to a master server at 127.6.153.254:43167
I20240501 14:08:48.240198 7121 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:48.240677 7121 heartbeater.cc:511] Master 127.6.153.254:43167 requested a full tablet report, sending...
I20240501 14:08:48.241559 6802 ts_manager.cc:194] Registered new tserver with Master: af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:48.242092 6802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.194:43127
W20240501 14:08:48.342152 7124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:48.342375 7124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:48.342430 7124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:48.345544 7124 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:48.345623 7124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:48.345705 7124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:08:48.349422 7124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43167
--never_fsync=true
--heap_profile_path=/tmp/kudu.7124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:48.350314 7124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:48.351473 7124 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:48.360055 7130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:48.360302 7129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:48.360633 7132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:48.360996 7124 server_base.cc:1034] running on GCE node
I20240501 14:08:48.361361 7124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:48.361900 7124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:48.363050 7124 hybrid_clock.cc:648] HybridClock initialized: now 1714572528363031 us; error 30 us; skew 500 ppm
I20240501 14:08:48.364817 7124 webserver.cc:460] Webserver started at http://127.6.153.195:41279/ using document root <none> and password file <none>
I20240501 14:08:48.365332 7124 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:48.365388 7124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:48.365556 7124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:48.367128 7124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "60b660863ceb4187bcb3e8e45f7a5594"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.367607 7124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "60b660863ceb4187bcb3e8e45f7a5594"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.370661 7124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 14:08:48.372715 7138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.373672 7124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:48.373771 7124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "60b660863ceb4187bcb3e8e45f7a5594"
format_stamp: "Formatted at 2024-05-01 14:08:48 on dist-test-slave-cmf0"
I20240501 14:08:48.373853 7124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:48.388330 7124 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:48.388948 7124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:48.389099 7124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:48.389598 7124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:48.390460 7124 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:48.390504 7124 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.390556 7124 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:48.390583 7124 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:48.398663 7124 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:40109
I20240501 14:08:48.398684 7251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:40109 every 8 connection(s)
I20240501 14:08:48.399585 7124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:08:48.403537 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 7124
I20240501 14:08:48.403626 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 14:08:48.407822 7252 heartbeater.cc:347] Connected to a master server at 127.6.153.254:43167
I20240501 14:08:48.408062 7252 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:48.408576 7252 heartbeater.cc:511] Master 127.6.153.254:43167 requested a full tablet report, sending...
I20240501 14:08:48.409466 6802 ts_manager.cc:194] Registered new tserver with Master: 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:48.409979 6802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:44205
I20240501 14:08:48.417517 6759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240501 14:08:48.427872 6759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240501 14:08:49.080605 6990 heartbeater.cc:503] Master 127.6.153.254:43167 was elected leader, sending a full tablet report...
I20240501 14:08:49.243957 7121 heartbeater.cc:503] Master 127.6.153.254:43167 was elected leader, sending a full tablet report...
I20240501 14:08:49.411986 7252 heartbeater.cc:503] Master 127.6.153.254:43167 was elected leader, sending a full tablet report...
I20240501 14:08:52.180480 6802 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37494:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\200\007\001\000\000\000\000\000\006\001\000\000\000\000\000\007\001\000\377\377\377\177"
indirect_data: "<redacted>"""
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 8
seed: 0
}
}
W20240501 14:08:52.183650 6802 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240501 14:08:52.204371 7050 tablet_service.cc:1467] Processing CreateTablet for tablet cc39e68201ff4ac2894f274498f92a4c (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.204622 6924 tablet_service.cc:1467] Processing CreateTablet for tablet 50f5cfb319f64c96bb4be1f9f7ada86a (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.204622 7185 tablet_service.cc:1467] Processing CreateTablet for tablet d1eae18765cd49e7a38bf85a6d1b40da (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.205271 7046 tablet_service.cc:1467] Processing CreateTablet for tablet 70df2f4c2e024234abf62fe8fea726f7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.205657 7050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc39e68201ff4ac2894f274498f92a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.205699 6924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f5cfb319f64c96bb4be1f9f7ada86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.205696 7184 tablet_service.cc:1467] Processing CreateTablet for tablet 310fbceabf1f40fe8bd8b72e17e99bd4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.205924 7183 tablet_service.cc:1467] Processing CreateTablet for tablet 06cac4dab02048839d0f5f0b989eae9c (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.206159 7047 tablet_service.cc:1467] Processing CreateTablet for tablet e8f09acc91cf4860ac51a8cf2b22c3fb (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.206331 7182 tablet_service.cc:1467] Processing CreateTablet for tablet c51190e645bf4e7e90ac3c3e044e9c53 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.206416 7047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f09acc91cf4860ac51a8cf2b22c3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.206724 7181 tablet_service.cc:1467] Processing CreateTablet for tablet cc39e68201ff4ac2894f274498f92a4c (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.206840 7048 tablet_service.cc:1467] Processing CreateTablet for tablet 5c307d6bcbc84de79f0933a2b3b4d8cd (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.207067 7048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c307d6bcbc84de79f0933a2b3b4d8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.207113 7180 tablet_service.cc:1467] Processing CreateTablet for tablet ed0c931c1560432c8a0ff141bb5adacd (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.204948 7045 tablet_service.cc:1467] Processing CreateTablet for tablet 99ea9113d14e467bbfd8fbb217aefe68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.207500 7179 tablet_service.cc:1467] Processing CreateTablet for tablet 5c307d6bcbc84de79f0933a2b3b4d8cd (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.207599 7045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ea9113d14e467bbfd8fbb217aefe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.207562 6923 tablet_service.cc:1467] Processing CreateTablet for tablet d1eae18765cd49e7a38bf85a6d1b40da (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.207733 7051 tablet_service.cc:1467] Processing CreateTablet for tablet c51190e645bf4e7e90ac3c3e044e9c53 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.207798 6923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eae18765cd49e7a38bf85a6d1b40da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.207942 7051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c51190e645bf4e7e90ac3c3e044e9c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.207907 7178 tablet_service.cc:1467] Processing CreateTablet for tablet e8f09acc91cf4860ac51a8cf2b22c3fb (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.207607 7049 tablet_service.cc:1467] Processing CreateTablet for tablet ed0c931c1560432c8a0ff141bb5adacd (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.208176 7049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0c931c1560432c8a0ff141bb5adacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.208308 6922 tablet_service.cc:1467] Processing CreateTablet for tablet 310fbceabf1f40fe8bd8b72e17e99bd4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.208300 7177 tablet_service.cc:1467] Processing CreateTablet for tablet 70df2f4c2e024234abf62fe8fea726f7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.208334 7052 tablet_service.cc:1467] Processing CreateTablet for tablet 06cac4dab02048839d0f5f0b989eae9c (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.204629 7044 tablet_service.cc:1467] Processing CreateTablet for tablet 34c9dabc6c5e4c75832a200effd27050 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.208513 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 310fbceabf1f40fe8bd8b72e17e99bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.208541 7052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06cac4dab02048839d0f5f0b989eae9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.208726 7044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c9dabc6c5e4c75832a200effd27050. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.208683 7176 tablet_service.cc:1467] Processing CreateTablet for tablet 99ea9113d14e467bbfd8fbb217aefe68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.208811 7053 tablet_service.cc:1467] Processing CreateTablet for tablet 310fbceabf1f40fe8bd8b72e17e99bd4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.209028 7053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 310fbceabf1f40fe8bd8b72e17e99bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.208998 6921 tablet_service.cc:1467] Processing CreateTablet for tablet 06cac4dab02048839d0f5f0b989eae9c (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.204371 7055 tablet_service.cc:1467] Processing CreateTablet for tablet 50f5cfb319f64c96bb4be1f9f7ada86a (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.209215 6921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06cac4dab02048839d0f5f0b989eae9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.209277 7055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f5cfb319f64c96bb4be1f9f7ada86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.209297 7186 tablet_service.cc:1467] Processing CreateTablet for tablet 50f5cfb319f64c96bb4be1f9f7ada86a (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.209380 7054 tablet_service.cc:1467] Processing CreateTablet for tablet d1eae18765cd49e7a38bf85a6d1b40da (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.205699 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eae18765cd49e7a38bf85a6d1b40da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.209576 7054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eae18765cd49e7a38bf85a6d1b40da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.209684 6920 tablet_service.cc:1467] Processing CreateTablet for tablet c51190e645bf4e7e90ac3c3e044e9c53 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.209910 6920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c51190e645bf4e7e90ac3c3e044e9c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.209935 7046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70df2f4c2e024234abf62fe8fea726f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.210358 6919 tablet_service.cc:1467] Processing CreateTablet for tablet cc39e68201ff4ac2894f274498f92a4c (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.210579 6919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc39e68201ff4ac2894f274498f92a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.211045 6918 tablet_service.cc:1467] Processing CreateTablet for tablet ed0c931c1560432c8a0ff141bb5adacd (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.211259 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0c931c1560432c8a0ff141bb5adacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.211434 7184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 310fbceabf1f40fe8bd8b72e17e99bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.211726 6917 tablet_service.cc:1467] Processing CreateTablet for tablet 5c307d6bcbc84de79f0933a2b3b4d8cd (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.211951 6917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c307d6bcbc84de79f0933a2b3b4d8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.212409 6916 tablet_service.cc:1467] Processing CreateTablet for tablet e8f09acc91cf4860ac51a8cf2b22c3fb (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.212580 7178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f09acc91cf4860ac51a8cf2b22c3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.212620 6916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f09acc91cf4860ac51a8cf2b22c3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.213114 6915 tablet_service.cc:1467] Processing CreateTablet for tablet 70df2f4c2e024234abf62fe8fea726f7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.213330 6915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70df2f4c2e024234abf62fe8fea726f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.213793 6914 tablet_service.cc:1467] Processing CreateTablet for tablet 99ea9113d14e467bbfd8fbb217aefe68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.214002 6914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ea9113d14e467bbfd8fbb217aefe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.214439 6913 tablet_service.cc:1467] Processing CreateTablet for tablet 34c9dabc6c5e4c75832a200effd27050 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.214655 6913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c9dabc6c5e4c75832a200effd27050. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.214751 7180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0c931c1560432c8a0ff141bb5adacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.216440 7179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c307d6bcbc84de79f0933a2b3b4d8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.218379 7181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc39e68201ff4ac2894f274498f92a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.220182 7182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c51190e645bf4e7e90ac3c3e044e9c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.221904 7176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ea9113d14e467bbfd8fbb217aefe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.223831 7183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06cac4dab02048839d0f5f0b989eae9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.224751 7177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70df2f4c2e024234abf62fe8fea726f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.225782 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f5cfb319f64c96bb4be1f9f7ada86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.229204 7272 tablet_bootstrap.cc:492] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.230506 7270 tablet_bootstrap.cc:492] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.230698 7271 tablet_bootstrap.cc:492] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.231576 7185 tablet_service.cc:1467] Processing CreateTablet for tablet 34c9dabc6c5e4c75832a200effd27050 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.231665 6918 tablet_service.cc:1467] Processing CreateTablet for tablet ff02acd1ff0b47c3aa559c4ff4991df6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.231824 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c9dabc6c5e4c75832a200effd27050. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.231906 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff02acd1ff0b47c3aa559c4ff4991df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.232944 7271 tablet_bootstrap.cc:654] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.234107 7272 tablet_bootstrap.cc:654] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.234287 7046 tablet_service.cc:1467] Processing CreateTablet for tablet ff02acd1ff0b47c3aa559c4ff4991df6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.234565 7046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff02acd1ff0b47c3aa559c4ff4991df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.234850 7271 log.cc:824] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:52.234850 7272 log.cc:824] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:52.236609 7271 tablet_bootstrap.cc:492] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.236609 7272 tablet_bootstrap.cc:492] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.236774 7271 ts_tablet_manager.cc:1367] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240501 14:08:52.236774 7272 ts_tablet_manager.cc:1367] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20240501 14:08:52.237313 7046 tablet_service.cc:1467] Processing CreateTablet for tablet 167ff34132cc46e5b900c6ca03d1951f (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.237540 7046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 167ff34132cc46e5b900c6ca03d1951f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.238934 7270 tablet_bootstrap.cc:654] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.238948 7185 tablet_service.cc:1467] Processing CreateTablet for tablet ff02acd1ff0b47c3aa559c4ff4991df6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.239192 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff02acd1ff0b47c3aa559c4ff4991df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.239717 7270 log.cc:824] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:52.240049 6918 tablet_service.cc:1467] Processing CreateTablet for tablet 167ff34132cc46e5b900c6ca03d1951f (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.240319 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 167ff34132cc46e5b900c6ca03d1951f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.241222 6924 tablet_service.cc:1467] Processing CreateTablet for tablet 0980e052557a46cf91cb652c15eef515 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.241344 6921 tablet_service.cc:1467] Processing CreateTablet for tablet 11558a225dae48ecbbd357a0b31ff2e8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.241464 6924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0980e052557a46cf91cb652c15eef515. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.241575 6921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11558a225dae48ecbbd357a0b31ff2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.241844 7185 tablet_service.cc:1467] Processing CreateTablet for tablet 167ff34132cc46e5b900c6ca03d1951f (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.242053 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 167ff34132cc46e5b900c6ca03d1951f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.242920 7272 raft_consensus.cc:348] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.243127 7272 raft_consensus.cc:374] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.243211 7272 raft_consensus.cc:729] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.243522 7178 tablet_service.cc:1467] Processing CreateTablet for tablet 0980e052557a46cf91cb652c15eef515 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.243779 7178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0980e052557a46cf91cb652c15eef515. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.243719 7272 consensus_queue.cc:260] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.243870 7179 tablet_service.cc:1467] Processing CreateTablet for tablet 11558a225dae48ecbbd357a0b31ff2e8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.244064 7179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11558a225dae48ecbbd357a0b31ff2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.244649 7272 ts_tablet_manager.cc:1398] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20240501 14:08:52.245021 7272 tablet_bootstrap.cc:492] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.246013 7272 tablet_bootstrap.cc:654] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.246511 7046 tablet_service.cc:1467] Processing CreateTablet for tablet 0980e052557a46cf91cb652c15eef515 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 14:08:52.246549 7047 tablet_service.cc:1467] Processing CreateTablet for tablet 11558a225dae48ecbbd357a0b31ff2e8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bd0486325dff43e18d0b6652e7fd1946]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 14:08:52.246802 7046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0980e052557a46cf91cb652c15eef515. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.247424 7272 tablet_bootstrap.cc:492] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.247548 7272 ts_tablet_manager.cc:1367] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:08:52.247670 7047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11558a225dae48ecbbd357a0b31ff2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:52.247753 7271 raft_consensus.cc:348] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.247969 7271 raft_consensus.cc:374] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.248026 7271 raft_consensus.cc:729] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.248374 7272 raft_consensus.cc:348] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.248522 7272 raft_consensus.cc:374] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.248601 7272 raft_consensus.cc:729] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.248764 7272 consensus_queue.cc:260] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.249132 7272 ts_tablet_manager.cc:1398] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.248463 7271 consensus_queue.cc:260] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.250207 7271 ts_tablet_manager.cc:1398] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20240501 14:08:52.250881 7270 tablet_bootstrap.cc:492] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.251149 7272 tablet_bootstrap.cc:492] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.251149 7270 ts_tablet_manager.cc:1367] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.000s
I20240501 14:08:52.251538 7271 tablet_bootstrap.cc:492] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.252221 7272 tablet_bootstrap.cc:654] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.252460 7271 tablet_bootstrap.cc:654] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.253444 7272 tablet_bootstrap.cc:492] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.253522 7272 ts_tablet_manager.cc:1367] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:52.253713 7271 tablet_bootstrap.cc:492] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.253794 7271 ts_tablet_manager.cc:1367] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:52.253939 7272 raft_consensus.cc:348] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.254034 7272 raft_consensus.cc:374] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.254062 7272 raft_consensus.cc:729] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.254192 7272 consensus_queue.cc:260] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.254318 7271 raft_consensus.cc:348] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.254410 7271 raft_consensus.cc:374] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.254437 7271 raft_consensus.cc:729] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.254449 7272 ts_tablet_manager.cc:1398] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.254539 7271 consensus_queue.cc:260] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.254597 7272 tablet_bootstrap.cc:492] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.254593 7270 raft_consensus.cc:348] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.254781 7271 ts_tablet_manager.cc:1398] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.254784 7270 raft_consensus.cc:374] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.254833 7270 raft_consensus.cc:729] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.254925 7271 tablet_bootstrap.cc:492] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.255384 7270 consensus_queue.cc:260] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.255770 7272 tablet_bootstrap.cc:654] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.255823 7271 tablet_bootstrap.cc:654] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.256278 7270 ts_tablet_manager.cc:1398] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20240501 14:08:52.257090 7270 tablet_bootstrap.cc:492] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.257297 7272 tablet_bootstrap.cc:492] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.257364 7272 ts_tablet_manager.cc:1367] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 14:08:52.257400 7271 tablet_bootstrap.cc:492] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.257464 7271 ts_tablet_manager.cc:1367] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:08:52.257757 7272 raft_consensus.cc:348] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.257845 7272 raft_consensus.cc:374] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.257875 7272 raft_consensus.cc:729] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.257844 7271 raft_consensus.cc:348] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.257937 7271 raft_consensus.cc:374] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.257967 7270 tablet_bootstrap.cc:654] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.257966 7271 raft_consensus.cc:729] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.257982 7272 consensus_queue.cc:260] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.258181 7271 consensus_queue.cc:260] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.258227 7272 ts_tablet_manager.cc:1398] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.258383 7272 tablet_bootstrap.cc:492] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.258445 7271 ts_tablet_manager.cc:1398] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.258589 7271 tablet_bootstrap.cc:492] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.259212 7272 tablet_bootstrap.cc:654] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.259706 7271 tablet_bootstrap.cc:654] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.260330 7270 tablet_bootstrap.cc:492] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.260421 7270 ts_tablet_manager.cc:1367] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:08:52.260685 7271 tablet_bootstrap.cc:492] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.260768 7271 ts_tablet_manager.cc:1367] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:52.260890 7272 tablet_bootstrap.cc:492] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.260969 7272 ts_tablet_manager.cc:1367] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 14:08:52.261168 7271 raft_consensus.cc:348] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.261266 7271 raft_consensus.cc:374] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.261307 7271 raft_consensus.cc:729] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.261325 7272 raft_consensus.cc:348] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.261418 7272 raft_consensus.cc:374] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.261453 7272 raft_consensus.cc:729] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.261417 7271 consensus_queue.cc:260] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.261567 7272 consensus_queue.cc:260] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.261672 7271 ts_tablet_manager.cc:1398] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.261798 7272 ts_tablet_manager.cc:1398] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:08:52.261833 7271 tablet_bootstrap.cc:492] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.261940 7272 tablet_bootstrap.cc:492] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.262661 7271 tablet_bootstrap.cc:654] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.262781 7272 tablet_bootstrap.cc:654] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.263689 7270 raft_consensus.cc:348] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.263813 7270 raft_consensus.cc:374] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.263847 7270 raft_consensus.cc:729] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.263847 7271 tablet_bootstrap.cc:492] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.263917 7271 ts_tablet_manager.cc:1367] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.263957 7270 consensus_queue.cc:260] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.264194 7270 ts_tablet_manager.cc:1398] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:08:52.264304 7271 raft_consensus.cc:348] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.264406 7271 raft_consensus.cc:374] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.264418 7270 tablet_bootstrap.cc:492] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.264446 7271 raft_consensus.cc:729] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.264555 7271 consensus_queue.cc:260] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.264802 7271 ts_tablet_manager.cc:1398] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.264950 7271 tablet_bootstrap.cc:492] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.265288 7270 tablet_bootstrap.cc:654] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.265774 7271 tablet_bootstrap.cc:654] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.266396 7272 tablet_bootstrap.cc:492] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.266479 7272 ts_tablet_manager.cc:1367] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240501 14:08:52.266876 7272 raft_consensus.cc:348] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.266963 7272 raft_consensus.cc:374] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.266989 7272 raft_consensus.cc:729] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.266990 7271 tablet_bootstrap.cc:492] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.267002 7270 tablet_bootstrap.cc:492] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.267061 7271 ts_tablet_manager.cc:1367] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.267066 7270 ts_tablet_manager.cc:1367] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:08:52.267103 7272 consensus_queue.cc:260] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.267470 7271 raft_consensus.cc:348] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.267490 7270 raft_consensus.cc:348] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.267561 7271 raft_consensus.cc:374] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.267576 7270 raft_consensus.cc:374] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.267593 7271 raft_consensus.cc:729] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.267606 7270 raft_consensus.cc:729] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.267691 7271 consensus_queue.cc:260] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.267714 7270 consensus_queue.cc:260] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.267926 7271 ts_tablet_manager.cc:1398] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.267974 7270 ts_tablet_manager.cc:1398] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.267975 7272 ts_tablet_manager.cc:1398] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:08:52.268066 7271 tablet_bootstrap.cc:492] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.268118 7272 tablet_bootstrap.cc:492] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.268118 7270 tablet_bootstrap.cc:492] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.268843 7271 tablet_bootstrap.cc:654] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.269073 7272 tablet_bootstrap.cc:654] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.269115 7270 tablet_bootstrap.cc:654] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.270301 7270 tablet_bootstrap.cc:492] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.270381 7270 ts_tablet_manager.cc:1367] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:52.270833 7270 raft_consensus.cc:348] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.270932 7270 raft_consensus.cc:374] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.270972 7270 raft_consensus.cc:729] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.271077 7270 consensus_queue.cc:260] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.271351 7270 ts_tablet_manager.cc:1398] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.271507 7270 tablet_bootstrap.cc:492] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.271711 7271 tablet_bootstrap.cc:492] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.271800 7271 ts_tablet_manager.cc:1367] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:08:52.272213 7271 raft_consensus.cc:348] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.272316 7271 raft_consensus.cc:374] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.272333 7272 tablet_bootstrap.cc:492] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.272349 7271 raft_consensus.cc:729] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.272372 7270 tablet_bootstrap.cc:654] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.272464 7272 ts_tablet_manager.cc:1367] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 14:08:52.272464 7271 consensus_queue.cc:260] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.272713 7271 ts_tablet_manager.cc:1398] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.272904 7271 tablet_bootstrap.cc:492] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.273432 7270 tablet_bootstrap.cc:492] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.273511 7270 ts_tablet_manager.cc:1367] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.273824 7271 tablet_bootstrap.cc:654] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.273908 7270 raft_consensus.cc:348] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.274001 7270 raft_consensus.cc:374] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.274035 7270 raft_consensus.cc:729] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.274143 7270 consensus_queue.cc:260] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.274379 7270 ts_tablet_manager.cc:1398] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.274086 7272 raft_consensus.cc:348] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.274549 7270 tablet_bootstrap.cc:492] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.274539 7272 raft_consensus.cc:374] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.274581 7272 raft_consensus.cc:729] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.274722 7272 consensus_queue.cc:260] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.275435 7270 tablet_bootstrap.cc:654] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.276397 7272 ts_tablet_manager.cc:1398] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240501 14:08:52.276628 7272 tablet_bootstrap.cc:492] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.276906 7271 tablet_bootstrap.cc:492] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.276976 7271 ts_tablet_manager.cc:1367] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 14:08:52.277189 7270 tablet_bootstrap.cc:492] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.277269 7270 ts_tablet_manager.cc:1367] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 14:08:52.277403 7271 raft_consensus.cc:348] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.277518 7271 raft_consensus.cc:374] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.277556 7271 raft_consensus.cc:729] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.277660 7271 consensus_queue.cc:260] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.277663 7270 raft_consensus.cc:348] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.277776 7270 raft_consensus.cc:374] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.277809 7270 raft_consensus.cc:729] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.277923 7270 consensus_queue.cc:260] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.277967 7271 ts_tablet_manager.cc:1398] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.278157 7270 ts_tablet_manager.cc:1398] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.278234 7271 tablet_bootstrap.cc:492] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.278303 7270 tablet_bootstrap.cc:492] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.279103 7271 tablet_bootstrap.cc:654] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.279286 7270 tablet_bootstrap.cc:654] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.280117 7272 tablet_bootstrap.cc:654] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.280139 7271 tablet_bootstrap.cc:492] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.280200 7271 ts_tablet_manager.cc:1367] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.280607 7271 raft_consensus.cc:348] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.280705 7271 raft_consensus.cc:374] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.280748 7271 raft_consensus.cc:729] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.280886 7271 consensus_queue.cc:260] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.281141 7271 ts_tablet_manager.cc:1398] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.281189 7270 tablet_bootstrap.cc:492] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.281262 7270 ts_tablet_manager.cc:1367] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:08:52.281303 7271 tablet_bootstrap.cc:492] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.281641 7270 raft_consensus.cc:348] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.281746 7270 raft_consensus.cc:374] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.281790 7270 raft_consensus.cc:729] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.281901 7270 consensus_queue.cc:260] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.282127 7271 tablet_bootstrap.cc:654] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.282145 7270 ts_tablet_manager.cc:1398] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.282308 7270 tablet_bootstrap.cc:492] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.283216 7270 tablet_bootstrap.cc:654] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.283854 7271 tablet_bootstrap.cc:492] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.283862 7272 tablet_bootstrap.cc:492] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.283933 7271 ts_tablet_manager.cc:1367] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:08:52.283942 7272 ts_tablet_manager.cc:1367] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240501 14:08:52.284197 7270 tablet_bootstrap.cc:492] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.284271 7270 ts_tablet_manager.cc:1367] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.284341 7271 raft_consensus.cc:348] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.284432 7271 raft_consensus.cc:374] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.284463 7271 raft_consensus.cc:729] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.284406 7272 raft_consensus.cc:348] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.284498 7272 raft_consensus.cc:374] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.284528 7272 raft_consensus.cc:729] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.284581 7271 consensus_queue.cc:260] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.284637 7272 consensus_queue.cc:260] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.284668 7270 raft_consensus.cc:348] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.284741 7270 raft_consensus.cc:374] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.284767 7270 raft_consensus.cc:729] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.284855 7270 consensus_queue.cc:260] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.284897 7271 ts_tablet_manager.cc:1398] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.284904 7272 ts_tablet_manager.cc:1398] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.285059 7272 tablet_bootstrap.cc:492] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.285096 7270 ts_tablet_manager.cc:1398] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.285120 7271 tablet_bootstrap.cc:492] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.285238 7270 tablet_bootstrap.cc:492] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.285908 7272 tablet_bootstrap.cc:654] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.286012 7270 tablet_bootstrap.cc:654] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.286278 7271 tablet_bootstrap.cc:654] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.287712 7270 tablet_bootstrap.cc:492] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.287798 7270 ts_tablet_manager.cc:1367] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:08:52.288230 7270 raft_consensus.cc:348] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.288348 7270 raft_consensus.cc:374] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.288381 7270 raft_consensus.cc:729] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.288503 7270 consensus_queue.cc:260] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.288794 7270 ts_tablet_manager.cc:1398] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.288844 7272 tablet_bootstrap.cc:492] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.288924 7272 ts_tablet_manager.cc:1367] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 14:08:52.288992 7270 tablet_bootstrap.cc:492] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.289326 7271 tablet_bootstrap.cc:492] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.289330 7272 raft_consensus.cc:348] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.289410 7271 ts_tablet_manager.cc:1367] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:08:52.289431 7272 raft_consensus.cc:374] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.289464 7272 raft_consensus.cc:729] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.289577 7272 consensus_queue.cc:260] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.289830 7272 ts_tablet_manager.cc:1398] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.289857 7270 tablet_bootstrap.cc:654] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.289809 7271 raft_consensus.cc:348] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.290020 7272 tablet_bootstrap.cc:492] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.290968 7272 tablet_bootstrap.cc:654] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.291136 7270 tablet_bootstrap.cc:492] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.291205 7270 ts_tablet_manager.cc:1367] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.291630 7270 raft_consensus.cc:348] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.291710 7271 raft_consensus.cc:374] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.291752 7270 raft_consensus.cc:374] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.291764 7271 raft_consensus.cc:729] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.291788 7270 raft_consensus.cc:729] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.291887 7270 consensus_queue.cc:260] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.291887 7271 consensus_queue.cc:260] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.292141 7270 ts_tablet_manager.cc:1398] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:08:52.292304 7270 tablet_bootstrap.cc:492] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.292551 7271 ts_tablet_manager.cc:1398] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:08:52.292711 7271 tablet_bootstrap.cc:492] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.293161 7270 tablet_bootstrap.cc:654] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.293545 7271 tablet_bootstrap.cc:654] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.294592 7271 tablet_bootstrap.cc:492] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.294592 7270 tablet_bootstrap.cc:492] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.294664 7271 ts_tablet_manager.cc:1367] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.294667 7270 ts_tablet_manager.cc:1367] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:08:52.295045 7271 raft_consensus.cc:348] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.295045 7270 raft_consensus.cc:348] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.295146 7270 raft_consensus.cc:374] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.295146 7271 raft_consensus.cc:374] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.295188 7271 raft_consensus.cc:729] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.295188 7270 raft_consensus.cc:729] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.295305 7270 consensus_queue.cc:260] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.295305 7271 consensus_queue.cc:260] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.295653 7270 ts_tablet_manager.cc:1398] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.295785 7271 ts_tablet_manager.cc:1398] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.295805 7270 tablet_bootstrap.cc:492] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.295962 7271 tablet_bootstrap.cc:492] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.296617 7270 tablet_bootstrap.cc:654] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.296730 7271 tablet_bootstrap.cc:654] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.297711 7272 tablet_bootstrap.cc:492] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.297725 7271 tablet_bootstrap.cc:492] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.297725 7270 tablet_bootstrap.cc:492] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.297796 7270 ts_tablet_manager.cc:1367] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.297793 7271 ts_tablet_manager.cc:1367] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.297802 7272 ts_tablet_manager.cc:1367] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240501 14:08:52.298173 7270 raft_consensus.cc:348] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.298257 7270 raft_consensus.cc:374] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.298288 7270 raft_consensus.cc:729] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.298275 7271 raft_consensus.cc:348] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.298254 7272 raft_consensus.cc:348] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.298357 7271 raft_consensus.cc:374] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.298362 7272 raft_consensus.cc:374] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.298385 7271 raft_consensus.cc:729] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.298390 7272 raft_consensus.cc:729] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.298399 7270 consensus_queue.cc:260] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.298485 7271 consensus_queue.cc:260] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.298496 7272 consensus_queue.cc:260] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.298663 7270 ts_tablet_manager.cc:1398] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.298698 7271 ts_tablet_manager.cc:1398] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.298758 7272 ts_tablet_manager.cc:1398] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:08:52.298817 7270 tablet_bootstrap.cc:492] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.298827 7271 tablet_bootstrap.cc:492] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.298910 7272 tablet_bootstrap.cc:492] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.299638 7271 tablet_bootstrap.cc:654] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.299638 7270 tablet_bootstrap.cc:654] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.300351 7278 raft_consensus.cc:482] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.300515 7278 raft_consensus.cc:504] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.300791 7270 tablet_bootstrap.cc:492] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.300820 7272 tablet_bootstrap.cc:654] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.300851 7270 ts_tablet_manager.cc:1367] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.302062 7270 raft_consensus.cc:348] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.302157 7270 raft_consensus.cc:374] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.302184 7270 raft_consensus.cc:729] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.302300 7270 consensus_queue.cc:260] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.302536 7270 ts_tablet_manager.cc:1398] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:08:52.302681 7270 tablet_bootstrap.cc:492] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.303632 7270 tablet_bootstrap.cc:654] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.301954 7278 leader_election.cc:288] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.305086 7272 tablet_bootstrap.cc:492] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.305166 7272 ts_tablet_manager.cc:1367] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:08:52.305491 7271 tablet_bootstrap.cc:492] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.305578 7271 ts_tablet_manager.cc:1367] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 14:08:52.305591 7272 raft_consensus.cc:348] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.305689 7272 raft_consensus.cc:374] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.305725 7272 raft_consensus.cc:729] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.305859 7272 consensus_queue.cc:260] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.306138 7272 ts_tablet_manager.cc:1398] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.306123 7271 raft_consensus.cc:348] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.306212 7271 raft_consensus.cc:374] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.306249 7271 raft_consensus.cc:729] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.306331 7272 tablet_bootstrap.cc:492] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.306381 7271 consensus_queue.cc:260] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.306631 7271 ts_tablet_manager.cc:1398] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.306785 7271 tablet_bootstrap.cc:492] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.307194 7272 tablet_bootstrap.cc:654] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.308378 7271 tablet_bootstrap.cc:654] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.309131 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c9dabc6c5e4c75832a200effd27050" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.309432 6944 raft_consensus.cc:2454] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.309474 7270 tablet_bootstrap.cc:492] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.309553 7270 ts_tablet_manager.cc:1367] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 14:08:52.309589 7281 raft_consensus.cc:482] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.309715 7281 raft_consensus.cc:504] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.309973 7270 raft_consensus.cc:348] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.309990 7142 leader_election.cc:302] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.310047 7270 raft_consensus.cc:374] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.310072 7270 raft_consensus.cc:729] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.310169 7270 consensus_queue.cc:260] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.310487 7270 ts_tablet_manager.cc:1398] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.310662 7270 tablet_bootstrap.cc:492] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.310914 7281 leader_election.cc:288] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.311540 7270 tablet_bootstrap.cc:654] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.312080 7278 raft_consensus.cc:2781] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.312156 7278 raft_consensus.cc:482] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.312216 7278 raft_consensus.cc:3037] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.313202 7278 raft_consensus.cc:504] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.313570 7278 leader_election.cc:288] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.313688 7271 tablet_bootstrap.cc:492] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.313766 7271 ts_tablet_manager.cc:1367] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 14:08:52.313850 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c9dabc6c5e4c75832a200effd27050" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.313956 6944 raft_consensus.cc:3037] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.314169 7271 raft_consensus.cc:348] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.314270 7271 raft_consensus.cc:374] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.314298 7271 raft_consensus.cc:729] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.314301 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c9dabc6c5e4c75832a200effd27050" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.314409 7271 consensus_queue.cc:260] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.314469 7075 raft_consensus.cc:2454] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.314654 7271 ts_tablet_manager.cc:1398] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.314806 7271 tablet_bootstrap.cc:492] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594: Bootstrap starting.
I20240501 14:08:52.315006 6944 raft_consensus.cc:2454] T 34c9dabc6c5e4c75832a200effd27050 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 1.
I20240501 14:08:52.315251 7142 leader_election.cc:302] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.315243 7074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c9dabc6c5e4c75832a200effd27050" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.315376 7074 raft_consensus.cc:3037] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.315862 7271 tablet_bootstrap.cc:654] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.316030 7278 raft_consensus.cc:2781] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.316119 7278 raft_consensus.cc:686] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 LEADER]: Becoming Leader. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Running, Role: LEADER
I20240501 14:08:52.316438 7278 consensus_queue.cc:237] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.316637 7074 raft_consensus.cc:2454] T 34c9dabc6c5e4c75832a200effd27050 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 1.
I20240501 14:08:52.318980 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f09acc91cf4860ac51a8cf2b22c3fb" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.319108 6944 raft_consensus.cc:2454] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.319300 7271 tablet_bootstrap.cc:492] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594: No bootstrap required, opened a new log
I20240501 14:08:52.319406 7271 ts_tablet_manager.cc:1367] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
W20240501 14:08:52.319581 6991 tablet.cc:2369] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:08:52.319594 7011 leader_election.cc:302] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.319852 7320 raft_consensus.cc:2781] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.319816 7271 raft_consensus.cc:348] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.319900 7271 raft_consensus.cc:374] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.319898 7320 raft_consensus.cc:482] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.319921 7271 raft_consensus.cc:729] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.319937 7320 raft_consensus.cc:3037] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.320003 7271 consensus_queue.cc:260] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.320248 7271 ts_tablet_manager.cc:1398] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.320823 7320 raft_consensus.cc:504] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.320968 7272 tablet_bootstrap.cc:492] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.321045 7272 ts_tablet_manager.cc:1367] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20240501 14:08:52.321066 7270 tablet_bootstrap.cc:492] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.321128 7270 ts_tablet_manager.cc:1367] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240501 14:08:52.321202 7320 leader_election.cc:288] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.321519 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f09acc91cf4860ac51a8cf2b22c3fb" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.321537 7270 raft_consensus.cc:348] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.321616 7270 raft_consensus.cc:374] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.321636 6944 raft_consensus.cc:3037] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.321645 7270 raft_consensus.cc:729] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.321763 7270 consensus_queue.cc:260] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.322016 7270 ts_tablet_manager.cc:1398] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.322171 7270 tablet_bootstrap.cc:492] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a: Bootstrap starting.
I20240501 14:08:52.322515 6944 raft_consensus.cc:2454] T e8f09acc91cf4860ac51a8cf2b22c3fb P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.322268 6802 catalog_manager.cc:5484] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "60b660863ceb4187bcb3e8e45f7a5594" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: HEALTHY } } }
I20240501 14:08:52.321516 7272 raft_consensus.cc:348] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.322893 7272 raft_consensus.cc:374] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.322923 7272 raft_consensus.cc:729] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.323037 7272 consensus_queue.cc:260] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.323107 7270 tablet_bootstrap.cc:654] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.323405 7272 ts_tablet_manager.cc:1398] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:08:52.323668 7272 tablet_bootstrap.cc:492] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.324808 7272 tablet_bootstrap.cc:654] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.325335 7011 leader_election.cc:302] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.325981 7270 tablet_bootstrap.cc:492] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a: No bootstrap required, opened a new log
I20240501 14:08:52.326071 7270 ts_tablet_manager.cc:1367] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:08:52.326500 7320 raft_consensus.cc:2781] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.326524 7270 raft_consensus.cc:348] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.326598 7320 raft_consensus.cc:686] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [term 1 LEADER]: Becoming Leader. State: Replica: af33d7febb294622a260df662f18821a, State: Running, Role: LEADER
I20240501 14:08:52.326617 7270 raft_consensus.cc:374] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.326647 7270 raft_consensus.cc:729] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af33d7febb294622a260df662f18821a, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.326766 7270 consensus_queue.cc:260] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.326912 7320 consensus_queue.cc:237] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.327147 7270 ts_tablet_manager.cc:1398] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.327898 7272 tablet_bootstrap.cc:492] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.327971 7272 ts_tablet_manager.cc:1367] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:08:52.328318 7272 raft_consensus.cc:348] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.328404 7272 raft_consensus.cc:374] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.328433 7272 raft_consensus.cc:729] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.328541 7272 consensus_queue.cc:260] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.328812 7272 ts_tablet_manager.cc:1398] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:08:52.329022 7272 tablet_bootstrap.cc:492] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.329068 7276 raft_consensus.cc:482] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.329145 7276 raft_consensus.cc:504] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.329932 7272 tablet_bootstrap.cc:654] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.330085 7276 leader_election.cc:288] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.330266 6802 catalog_manager.cc:5484] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a reported cstate change: term changed from 0 to 1, leader changed from <none> to af33d7febb294622a260df662f18821a (127.6.153.194). New cstate: current_term: 1 leader_uuid: "af33d7febb294622a260df662f18821a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.333133 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f09acc91cf4860ac51a8cf2b22c3fb" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.333237 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f09acc91cf4860ac51a8cf2b22c3fb" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.333315 7205 raft_consensus.cc:3037] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.333803 7276 raft_consensus.cc:482] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.333886 7276 raft_consensus.cc:504] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.334208 7276 leader_election.cc:288] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.334211 7205 raft_consensus.cc:2454] T e8f09acc91cf4860ac51a8cf2b22c3fb P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.335420 7272 tablet_bootstrap.cc:492] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.335517 7272 ts_tablet_manager.cc:1367] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240501 14:08:52.335978 7272 raft_consensus.cc:348] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.336084 7272 raft_consensus.cc:374] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.336115 7272 raft_consensus.cc:729] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.336268 7272 consensus_queue.cc:260] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.336560 7272 ts_tablet_manager.cc:1398] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.336778 7272 tablet_bootstrap.cc:492] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.337735 7272 tablet_bootstrap.cc:654] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.338853 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eae18765cd49e7a38bf85a6d1b40da" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.338996 7205 raft_consensus.cc:2454] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.339004 7074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eae18765cd49e7a38bf85a6d1b40da" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.339130 7074 raft_consensus.cc:2454] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.339195 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f5cfb319f64c96bb4be1f9f7ada86a" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.339300 7206 raft_consensus.cc:2454] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.339319 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f5cfb319f64c96bb4be1f9f7ada86a" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.339393 6878 leader_election.cc:302] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.339452 7075 raft_consensus.cc:2454] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.339506 7272 tablet_bootstrap.cc:492] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.339577 7272 ts_tablet_manager.cc:1367] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:08:52.339670 7276 raft_consensus.cc:2781] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.339721 7276 raft_consensus.cc:482] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.339761 7276 raft_consensus.cc:3037] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.339963 6877 leader_election.cc:302] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.340024 7272 raft_consensus.cc:348] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.340121 7272 raft_consensus.cc:374] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.340157 7272 raft_consensus.cc:729] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.340289 7272 consensus_queue.cc:260] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.340606 7336 raft_consensus.cc:2781] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.340636 7272 ts_tablet_manager.cc:1398] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.340682 7276 raft_consensus.cc:504] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.340884 7272 tablet_bootstrap.cc:492] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e: Bootstrap starting.
I20240501 14:08:52.341077 7276 leader_election.cc:288] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Requested vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.341399 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eae18765cd49e7a38bf85a6d1b40da" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.341507 7075 raft_consensus.cc:3037] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.341841 7272 tablet_bootstrap.cc:654] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:52.342490 7075 raft_consensus.cc:2454] T d1eae18765cd49e7a38bf85a6d1b40da P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.342751 6877 leader_election.cc:302] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.342922 7272 tablet_bootstrap.cc:492] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e: No bootstrap required, opened a new log
I20240501 14:08:52.342928 7335 raft_consensus.cc:2781] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.343024 7272 ts_tablet_manager.cc:1367] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:52.343034 7335 raft_consensus.cc:686] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [term 1 LEADER]: Becoming Leader. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Running, Role: LEADER
I20240501 14:08:52.340720 7336 raft_consensus.cc:482] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.343097 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eae18765cd49e7a38bf85a6d1b40da" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.343220 7206 raft_consensus.cc:3037] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.343207 7336 raft_consensus.cc:3037] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.343513 7335 consensus_queue.cc:237] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.343540 7272 raft_consensus.cc:348] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.343619 7272 raft_consensus.cc:374] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:52.343645 7272 raft_consensus.cc:729] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Initialized, Role: FOLLOWER
I20240501 14:08:52.343762 7272 consensus_queue.cc:260] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.344012 7272 ts_tablet_manager.cc:1398] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:08:52.344233 7206 raft_consensus.cc:2454] T d1eae18765cd49e7a38bf85a6d1b40da P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.344359 7336 raft_consensus.cc:504] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.344772 7336 leader_election.cc:288] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Requested vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.345145 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f5cfb319f64c96bb4be1f9f7ada86a" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.345240 7206 raft_consensus.cc:3037] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.345582 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f5cfb319f64c96bb4be1f9f7ada86a" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.345685 7075 raft_consensus.cc:3037] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.346104 7206 raft_consensus.cc:2454] T 50f5cfb319f64c96bb4be1f9f7ada86a P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.346383 6878 leader_election.cc:302] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.346530 7075 raft_consensus.cc:2454] T 50f5cfb319f64c96bb4be1f9f7ada86a P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.346812 7276 raft_consensus.cc:2781] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.346896 7276 raft_consensus.cc:686] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [term 1 LEADER]: Becoming Leader. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Running, Role: LEADER
I20240501 14:08:52.347016 7276 consensus_queue.cc:237] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.347154 6802 catalog_manager.cc:5484] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193). New cstate: current_term: 1 leader_uuid: "9c07440c3b8d4a939f56e878ef954d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.349023 7281 raft_consensus.cc:482] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.349115 7281 raft_consensus.cc:504] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.349457 7281 leader_election.cc:288] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.349756 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "167ff34132cc46e5b900c6ca03d1951f" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.349866 6944 raft_consensus.cc:2454] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.349864 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "167ff34132cc46e5b900c6ca03d1951f" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.349952 7206 raft_consensus.cc:2454] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.350133 7011 leader_election.cc:302] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.350302 7281 raft_consensus.cc:2781] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.350354 7281 raft_consensus.cc:482] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.350396 7281 raft_consensus.cc:3037] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.350725 6802 catalog_manager.cc:5484] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193). New cstate: current_term: 1 leader_uuid: "9c07440c3b8d4a939f56e878ef954d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.351303 7281 raft_consensus.cc:504] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.351637 7281 leader_election.cc:288] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.351946 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "167ff34132cc46e5b900c6ca03d1951f" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.352051 6944 raft_consensus.cc:3037] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.352157 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "167ff34132cc46e5b900c6ca03d1951f" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.352258 7206 raft_consensus.cc:3037] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.352699 6944 raft_consensus.cc:2454] T 167ff34132cc46e5b900c6ca03d1951f P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.352921 7206 raft_consensus.cc:2454] T 167ff34132cc46e5b900c6ca03d1951f P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.352964 7011 leader_election.cc:302] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.353168 7281 raft_consensus.cc:2781] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.353245 7281 raft_consensus.cc:686] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [term 1 LEADER]: Becoming Leader. State: Replica: af33d7febb294622a260df662f18821a, State: Running, Role: LEADER
I20240501 14:08:52.353363 7281 consensus_queue.cc:237] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.354938 6802 catalog_manager.cc:5484] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a reported cstate change: term changed from 0 to 1, leader changed from <none> to af33d7febb294622a260df662f18821a (127.6.153.194). New cstate: current_term: 1 leader_uuid: "af33d7febb294622a260df662f18821a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.359624 7278 raft_consensus.cc:482] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.359709 7278 raft_consensus.cc:504] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.359772 7335 raft_consensus.cc:482] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.359843 7335 raft_consensus.cc:504] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.360080 7278 leader_election.cc:288] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.360164 7335 leader_election.cc:288] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.360438 7074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9113d14e467bbfd8fbb217aefe68" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.360528 7320 raft_consensus.cc:482] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.360549 7074 raft_consensus.cc:2454] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.360572 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9113d14e467bbfd8fbb217aefe68" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.360649 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9113d14e467bbfd8fbb217aefe68" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.360642 7320 raft_consensus.cc:504] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.360674 7206 raft_consensus.cc:2454] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.360750 6944 raft_consensus.cc:2454] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.360949 7320 leader_election.cc:288] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.360981 6877 leader_election.cc:302] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.361018 7142 leader_election.cc:302] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.361157 7278 raft_consensus.cc:2781] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.361224 7278 raft_consensus.cc:482] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.361222 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9113d14e467bbfd8fbb217aefe68" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.361270 7278 raft_consensus.cc:3037] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.361310 7075 raft_consensus.cc:2454] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.361402 7335 raft_consensus.cc:2781] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.361454 7335 raft_consensus.cc:482] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.361490 7335 raft_consensus.cc:3037] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.361491 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11558a225dae48ecbbd357a0b31ff2e8" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.361409 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11558a225dae48ecbbd357a0b31ff2e8" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.361634 6944 raft_consensus.cc:2454] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.361660 7206 raft_consensus.cc:2454] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.361886 7011 leader_election.cc:302] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.362038 7320 raft_consensus.cc:2781] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.362087 7320 raft_consensus.cc:482] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.362129 7320 raft_consensus.cc:3037] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.362228 7278 raft_consensus.cc:504] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.362416 7335 raft_consensus.cc:504] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.362537 7278 leader_election.cc:288] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.362946 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9113d14e467bbfd8fbb217aefe68" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.362985 7320 raft_consensus.cc:504] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.363101 6944 raft_consensus.cc:2379] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60b660863ceb4187bcb3e8e45f7a5594 in current term 1: Already voted for candidate 9c07440c3b8d4a939f56e878ef954d5e in this term.
I20240501 14:08:52.363204 7074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9113d14e467bbfd8fbb217aefe68" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.363296 7074 raft_consensus.cc:3037] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.363307 7320 leader_election.cc:288] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.363579 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11558a225dae48ecbbd357a0b31ff2e8" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.363667 6944 raft_consensus.cc:3037] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.363783 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11558a225dae48ecbbd357a0b31ff2e8" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.363857 7206 raft_consensus.cc:3037] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.364132 7074 raft_consensus.cc:2454] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.364320 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9113d14e467bbfd8fbb217aefe68" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.364456 7075 raft_consensus.cc:2379] T 99ea9113d14e467bbfd8fbb217aefe68 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 60b660863ceb4187bcb3e8e45f7a5594 in current term 1: Already voted for candidate 9c07440c3b8d4a939f56e878ef954d5e in this term.
I20240501 14:08:52.364437 6877 leader_election.cc:302] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.364477 6944 raft_consensus.cc:2454] T 11558a225dae48ecbbd357a0b31ff2e8 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.364511 7206 raft_consensus.cc:2454] T 11558a225dae48ecbbd357a0b31ff2e8 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.364590 7276 raft_consensus.cc:2781] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.364655 7276 raft_consensus.cc:686] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 LEADER]: Becoming Leader. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Running, Role: LEADER
I20240501 14:08:52.364784 7276 consensus_queue.cc:237] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.364807 7009 leader_election.cc:302] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.364960 7320 raft_consensus.cc:2781] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.365007 7139 leader_election.cc:302] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594; no voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a
I20240501 14:08:52.365043 7320 raft_consensus.cc:686] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [term 1 LEADER]: Becoming Leader. State: Replica: af33d7febb294622a260df662f18821a, State: Running, Role: LEADER
I20240501 14:08:52.365149 7320 consensus_queue.cc:237] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.365221 7278 raft_consensus.cc:2726] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 14:08:52.365756 7335 leader_election.cc:288] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Requested vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.365911 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9113d14e467bbfd8fbb217aefe68" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.366042 7206 raft_consensus.cc:2379] T 99ea9113d14e467bbfd8fbb217aefe68 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9c07440c3b8d4a939f56e878ef954d5e in current term 1: Already voted for candidate 60b660863ceb4187bcb3e8e45f7a5594 in this term.
I20240501 14:08:52.366623 6802 catalog_manager.cc:5484] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193). New cstate: current_term: 1 leader_uuid: "9c07440c3b8d4a939f56e878ef954d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.366914 6800 catalog_manager.cc:5484] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a reported cstate change: term changed from 0 to 1, leader changed from <none> to af33d7febb294622a260df662f18821a (127.6.153.194). New cstate: current_term: 1 leader_uuid: "af33d7febb294622a260df662f18821a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.375017 7335 raft_consensus.cc:482] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.375115 7335 raft_consensus.cc:504] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.375490 7335 leader_election.cc:288] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.375838 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c51190e645bf4e7e90ac3c3e044e9c53" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.375864 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c51190e645bf4e7e90ac3c3e044e9c53" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.375959 7206 raft_consensus.cc:2454] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.375967 7075 raft_consensus.cc:2454] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.376204 6878 leader_election.cc:302] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.376377 7335 raft_consensus.cc:2781] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.376422 7335 raft_consensus.cc:482] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.376447 7335 raft_consensus.cc:3037] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.377168 7335 raft_consensus.cc:504] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.377461 7335 leader_election.cc:288] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Requested vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.377748 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c51190e645bf4e7e90ac3c3e044e9c53" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.377776 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c51190e645bf4e7e90ac3c3e044e9c53" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.377843 7075 raft_consensus.cc:3037] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.377871 7206 raft_consensus.cc:3037] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.378490 7075 raft_consensus.cc:2454] T c51190e645bf4e7e90ac3c3e044e9c53 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.378545 7206 raft_consensus.cc:2454] T c51190e645bf4e7e90ac3c3e044e9c53 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.378729 6877 leader_election.cc:302] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.378866 7335 raft_consensus.cc:2781] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.378962 7335 raft_consensus.cc:686] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 LEADER]: Becoming Leader. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Running, Role: LEADER
I20240501 14:08:52.379097 7335 consensus_queue.cc:237] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.380637 6800 catalog_manager.cc:5484] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193). New cstate: current_term: 1 leader_uuid: "9c07440c3b8d4a939f56e878ef954d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.387735 7335 raft_consensus.cc:482] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.387847 7335 raft_consensus.cc:504] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.388195 7335 leader_election.cc:288] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.388597 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "310fbceabf1f40fe8bd8b72e17e99bd4" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.388602 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "310fbceabf1f40fe8bd8b72e17e99bd4" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.388711 7075 raft_consensus.cc:2454] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.388732 7206 raft_consensus.cc:2454] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.388978 6877 leader_election.cc:302] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.389148 7335 raft_consensus.cc:2781] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.389205 7335 raft_consensus.cc:482] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.389237 7335 raft_consensus.cc:3037] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.389981 7335 raft_consensus.cc:504] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.390298 7335 leader_election.cc:288] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Requested vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.390622 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "310fbceabf1f40fe8bd8b72e17e99bd4" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.390641 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "310fbceabf1f40fe8bd8b72e17e99bd4" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.390733 7075 raft_consensus.cc:3037] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.390749 7206 raft_consensus.cc:3037] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.391471 7206 raft_consensus.cc:2454] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.391505 7075 raft_consensus.cc:2454] T 310fbceabf1f40fe8bd8b72e17e99bd4 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.391747 6878 leader_election.cc:302] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.391970 7335 raft_consensus.cc:2781] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.392048 7335 raft_consensus.cc:686] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 LEADER]: Becoming Leader. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Running, Role: LEADER
I20240501 14:08:52.392168 7335 consensus_queue.cc:237] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.393791 6800 catalog_manager.cc:5484] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193). New cstate: current_term: 1 leader_uuid: "9c07440c3b8d4a939f56e878ef954d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
W20240501 14:08:52.403203 7253 tablet.cc:2369] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:08:52.437893 7335 raft_consensus.cc:482] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.438040 7335 raft_consensus.cc:504] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.438453 7335 leader_election.cc:288] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.438982 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0c931c1560432c8a0ff141bb5adacd" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.438975 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0c931c1560432c8a0ff141bb5adacd" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.439113 7206 raft_consensus.cc:2454] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.439117 7075 raft_consensus.cc:2454] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.439445 6878 leader_election.cc:302] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.439608 7335 raft_consensus.cc:2781] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.439666 7335 raft_consensus.cc:482] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.439702 7335 raft_consensus.cc:3037] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.440485 7335 raft_consensus.cc:504] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.440834 7335 leader_election.cc:288] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Requested vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.441197 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0c931c1560432c8a0ff141bb5adacd" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.441226 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0c931c1560432c8a0ff141bb5adacd" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.441324 7206 raft_consensus.cc:3037] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.441337 7075 raft_consensus.cc:3037] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.442008 7206 raft_consensus.cc:2454] T ed0c931c1560432c8a0ff141bb5adacd P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.442052 7075 raft_consensus.cc:2454] T ed0c931c1560432c8a0ff141bb5adacd P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.442291 6878 leader_election.cc:302] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.442451 7335 raft_consensus.cc:2781] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.442539 7335 raft_consensus.cc:686] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [term 1 LEADER]: Becoming Leader. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Running, Role: LEADER
I20240501 14:08:52.442672 7335 consensus_queue.cc:237] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.444429 6800 catalog_manager.cc:5484] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193). New cstate: current_term: 1 leader_uuid: "9c07440c3b8d4a939f56e878ef954d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.460677 7320 raft_consensus.cc:482] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.460826 7320 raft_consensus.cc:504] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.461055 7281 raft_consensus.cc:482] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.461143 7281 raft_consensus.cc:504] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.461362 7320 leader_election.cc:288] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.461410 7281 leader_election.cc:288] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.461809 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c307d6bcbc84de79f0933a2b3b4d8cd" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.461809 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c307d6bcbc84de79f0933a2b3b4d8cd" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.461856 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06cac4dab02048839d0f5f0b989eae9c" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.461915 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06cac4dab02048839d0f5f0b989eae9c" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.461948 6944 raft_consensus.cc:2454] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.461948 7206 raft_consensus.cc:2454] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.461975 7205 raft_consensus.cc:2454] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.462013 6943 raft_consensus.cc:2454] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 0.
I20240501 14:08:52.462230 7009 leader_election.cc:302] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.462373 7011 leader_election.cc:302] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.462409 7281 raft_consensus.cc:2781] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.462468 7281 raft_consensus.cc:482] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.462509 7281 raft_consensus.cc:3037] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.462606 7320 raft_consensus.cc:2781] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.462651 7320 raft_consensus.cc:482] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.462692 7320 raft_consensus.cc:3037] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.463618 7320 raft_consensus.cc:504] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.463663 7281 raft_consensus.cc:504] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.464028 7320 leader_election.cc:288] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.464030 7281 leader_election.cc:288] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.464432 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c307d6bcbc84de79f0933a2b3b4d8cd" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.464532 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06cac4dab02048839d0f5f0b989eae9c" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.464653 6944 raft_consensus.cc:3037] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.464712 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06cac4dab02048839d0f5f0b989eae9c" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.464768 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c307d6bcbc84de79f0933a2b3b4d8cd" candidate_uuid: "af33d7febb294622a260df662f18821a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.464569 6943 raft_consensus.cc:3037] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.464807 7205 raft_consensus.cc:3037] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.464883 7206 raft_consensus.cc:3037] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.465662 6944 raft_consensus.cc:2454] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.465662 7206 raft_consensus.cc:2454] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.465956 7009 leader_election.cc:302] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.466185 7205 raft_consensus.cc:2454] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.466185 6943 raft_consensus.cc:2454] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af33d7febb294622a260df662f18821a in term 1.
I20240501 14:08:52.466368 7281 raft_consensus.cc:2781] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.466451 7281 raft_consensus.cc:686] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [term 1 LEADER]: Becoming Leader. State: Replica: af33d7febb294622a260df662f18821a, State: Running, Role: LEADER
I20240501 14:08:52.466452 7009 leader_election.cc:302] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.466607 7281 consensus_queue.cc:237] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.466667 7320 raft_consensus.cc:2781] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.466771 7320 raft_consensus.cc:686] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [term 1 LEADER]: Becoming Leader. State: Replica: af33d7febb294622a260df662f18821a, State: Running, Role: LEADER
I20240501 14:08:52.466867 7320 consensus_queue.cc:237] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.468472 6800 catalog_manager.cc:5484] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a reported cstate change: term changed from 0 to 1, leader changed from <none> to af33d7febb294622a260df662f18821a (127.6.153.194). New cstate: current_term: 1 leader_uuid: "af33d7febb294622a260df662f18821a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.471634 6800 catalog_manager.cc:5484] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a reported cstate change: term changed from 0 to 1, leader changed from <none> to af33d7febb294622a260df662f18821a (127.6.153.194). New cstate: current_term: 1 leader_uuid: "af33d7febb294622a260df662f18821a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.482831 7335 raft_consensus.cc:482] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.482966 7335 raft_consensus.cc:504] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.483462 7335 leader_election.cc:288] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.483861 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980e052557a46cf91cb652c15eef515" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.483872 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980e052557a46cf91cb652c15eef515" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594" is_pre_election: true
I20240501 14:08:52.483999 7075 raft_consensus.cc:2454] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.484002 7205 raft_consensus.cc:2454] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 0.
I20240501 14:08:52.484287 6878 leader_election.cc:302] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.484463 7335 raft_consensus.cc:2781] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.484520 7335 raft_consensus.cc:482] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.484555 7335 raft_consensus.cc:3037] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
W20240501 14:08:52.485255 7122 tablet.cc:2369] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:08:52.485378 7335 raft_consensus.cc:504] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.485736 7335 leader_election.cc:288] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Requested vote from peers af33d7febb294622a260df662f18821a (127.6.153.194:34591), 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195:40109)
I20240501 14:08:52.486065 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980e052557a46cf91cb652c15eef515" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.486184 7075 raft_consensus.cc:3037] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.486276 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0980e052557a46cf91cb652c15eef515" candidate_uuid: "9c07440c3b8d4a939f56e878ef954d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b660863ceb4187bcb3e8e45f7a5594"
I20240501 14:08:52.486387 7205 raft_consensus.cc:3037] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.487135 7075 raft_consensus.cc:2454] T 0980e052557a46cf91cb652c15eef515 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.487205 7205 raft_consensus.cc:2454] T 0980e052557a46cf91cb652c15eef515 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c07440c3b8d4a939f56e878ef954d5e in term 1.
I20240501 14:08:52.487476 6878 leader_election.cc:302] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.487650 7335 raft_consensus.cc:2781] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.487735 7335 raft_consensus.cc:686] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 LEADER]: Becoming Leader. State: Replica: 9c07440c3b8d4a939f56e878ef954d5e, State: Running, Role: LEADER
I20240501 14:08:52.487874 7335 consensus_queue.cc:237] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.489499 6800 catalog_manager.cc:5484] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193). New cstate: current_term: 1 leader_uuid: "9c07440c3b8d4a939f56e878ef954d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:52.527472 7278 raft_consensus.cc:482] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.527630 7278 raft_consensus.cc:504] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.528074 7278 leader_election.cc:288] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.528581 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc39e68201ff4ac2894f274498f92a4c" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.528584 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc39e68201ff4ac2894f274498f92a4c" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.528725 7075 raft_consensus.cc:2454] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.528736 6943 raft_consensus.cc:2454] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.529011 7139 leader_election.cc:302] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.529177 7278 raft_consensus.cc:2781] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.529278 7278 raft_consensus.cc:482] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.529326 7278 raft_consensus.cc:3037] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.530086 7278 raft_consensus.cc:504] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.530436 7278 leader_election.cc:288] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.530768 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc39e68201ff4ac2894f274498f92a4c" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.530896 7075 raft_consensus.cc:3037] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.530858 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc39e68201ff4ac2894f274498f92a4c" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.530983 6943 raft_consensus.cc:3037] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.531637 7075 raft_consensus.cc:2454] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 1.
I20240501 14:08:52.531796 6943 raft_consensus.cc:2454] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 1.
I20240501 14:08:52.531912 7139 leader_election.cc:302] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.532070 7278 raft_consensus.cc:2781] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.532150 7278 raft_consensus.cc:686] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 LEADER]: Becoming Leader. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Running, Role: LEADER
I20240501 14:08:52.532281 7278 consensus_queue.cc:237] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.534142 6800 catalog_manager.cc:5484] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "60b660863ceb4187bcb3e8e45f7a5594" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: HEALTHY } } }
I20240501 14:08:52.563961 7278 raft_consensus.cc:482] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.564095 7278 raft_consensus.cc:504] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.564499 7278 leader_election.cc:288] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.565032 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff02acd1ff0b47c3aa559c4ff4991df6" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.565086 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff02acd1ff0b47c3aa559c4ff4991df6" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.565182 6943 raft_consensus.cc:2454] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.565268 7075 raft_consensus.cc:2454] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.565544 7142 leader_election.cc:302] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, 9c07440c3b8d4a939f56e878ef954d5e; no voters:
I20240501 14:08:52.565739 7278 raft_consensus.cc:2781] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.565799 7278 raft_consensus.cc:482] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.565834 7278 raft_consensus.cc:3037] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.566599 7278 raft_consensus.cc:504] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.566955 7278 leader_election.cc:288] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.567317 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff02acd1ff0b47c3aa559c4ff4991df6" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.567317 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff02acd1ff0b47c3aa559c4ff4991df6" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.567467 6943 raft_consensus.cc:3037] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.567467 7075 raft_consensus.cc:3037] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.568388 6943 raft_consensus.cc:2454] T ff02acd1ff0b47c3aa559c4ff4991df6 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 1.
I20240501 14:08:52.568388 7075 raft_consensus.cc:2454] T ff02acd1ff0b47c3aa559c4ff4991df6 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 1.
I20240501 14:08:52.568727 7139 leader_election.cc:302] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.568964 7278 raft_consensus.cc:2781] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.569057 7278 raft_consensus.cc:686] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 LEADER]: Becoming Leader. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Running, Role: LEADER
I20240501 14:08:52.569233 7278 consensus_queue.cc:237] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.570969 6800 catalog_manager.cc:5484] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "60b660863ceb4187bcb3e8e45f7a5594" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: HEALTHY } } }
I20240501 14:08:52.620631 7278 raft_consensus.cc:482] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:52.620767 7278 raft_consensus.cc:504] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.621182 7278 leader_election.cc:288] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.621637 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70df2f4c2e024234abf62fe8fea726f7" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a" is_pre_election: true
I20240501 14:08:52.621645 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70df2f4c2e024234abf62fe8fea726f7" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e" is_pre_election: true
I20240501 14:08:52.621774 6943 raft_consensus.cc:2454] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.621779 7075 raft_consensus.cc:2454] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 0.
I20240501 14:08:52.622061 7139 leader_election.cc:302] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.622237 7278 raft_consensus.cc:2781] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:52.622290 7278 raft_consensus.cc:482] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:52.622314 7278 raft_consensus.cc:3037] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.623059 7278 raft_consensus.cc:504] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.623435 7278 leader_election.cc:288] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Requested vote from peers 9c07440c3b8d4a939f56e878ef954d5e (127.6.153.193:34157), af33d7febb294622a260df662f18821a (127.6.153.194:34591)
I20240501 14:08:52.623772 7075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70df2f4c2e024234abf62fe8fea726f7" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af33d7febb294622a260df662f18821a"
I20240501 14:08:52.623781 6943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70df2f4c2e024234abf62fe8fea726f7" candidate_uuid: "60b660863ceb4187bcb3e8e45f7a5594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c07440c3b8d4a939f56e878ef954d5e"
I20240501 14:08:52.623872 7075 raft_consensus.cc:3037] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.623886 6943 raft_consensus.cc:3037] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:52.624547 7075 raft_consensus.cc:2454] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 1.
I20240501 14:08:52.624629 6943 raft_consensus.cc:2454] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b660863ceb4187bcb3e8e45f7a5594 in term 1.
I20240501 14:08:52.624809 7139 leader_election.cc:302] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60b660863ceb4187bcb3e8e45f7a5594, af33d7febb294622a260df662f18821a; no voters:
I20240501 14:08:52.624956 7278 raft_consensus.cc:2781] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:52.625041 7278 raft_consensus.cc:686] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 LEADER]: Becoming Leader. State: Replica: 60b660863ceb4187bcb3e8e45f7a5594, State: Running, Role: LEADER
I20240501 14:08:52.625198 7278 consensus_queue.cc:237] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } }
I20240501 14:08:52.626792 6800 catalog_manager.cc:5484] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60b660863ceb4187bcb3e8e45f7a5594 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "60b660863ceb4187bcb3e8e45f7a5594" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 } health_report { overall_health: HEALTHY } } }
I20240501 14:08:52.726260 7278 consensus_queue.cc:1035] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.726325 7335 consensus_queue.cc:1035] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.731432 7278 consensus_queue.cc:1035] T 34c9dabc6c5e4c75832a200effd27050 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.732376 7276 consensus_queue.cc:1035] T 50f5cfb319f64c96bb4be1f9f7ada86a P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.736615 7328 consensus_queue.cc:1035] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.740222 7320 consensus_queue.cc:1035] T e8f09acc91cf4860ac51a8cf2b22c3fb P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.779704 7320 consensus_queue.cc:1035] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.782804 7320 consensus_queue.cc:1035] T 11558a225dae48ecbbd357a0b31ff2e8 P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 14:08:52.783743 7353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:52.784106 7353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:52.787648 7353 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 14:08:52.802593 7276 consensus_queue.cc:1035] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.806159 7335 consensus_queue.cc:1035] T c51190e645bf4e7e90ac3c3e044e9c53 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_0'
I20240501 14:08:52.825095 7335 consensus_queue.cc:1035] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.836784 7335 consensus_queue.cc:1035] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.840909 7276 consensus_queue.cc:1035] T d1eae18765cd49e7a38bf85a6d1b40da P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.844185 7336 consensus_queue.cc:1035] T 99ea9113d14e467bbfd8fbb217aefe68 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.848043 7281 consensus_queue.cc:1035] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.857554 7320 consensus_queue.cc:1035] T 167ff34132cc46e5b900c6ca03d1951f P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.884763 7075 raft_consensus.cc:1261] T cc39e68201ff4ac2894f274498f92a4c P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Refusing update from remote peer 60b660863ceb4187bcb3e8e45f7a5594: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:08:52.885771 6944 raft_consensus.cc:1261] T cc39e68201ff4ac2894f274498f92a4c P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Refusing update from remote peer 60b660863ceb4187bcb3e8e45f7a5594: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:08:52.886888 6943 raft_consensus.cc:1261] T 70df2f4c2e024234abf62fe8fea726f7 P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Refusing update from remote peer 60b660863ceb4187bcb3e8e45f7a5594: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:08:52.887059 6942 raft_consensus.cc:1261] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Refusing update from remote peer af33d7febb294622a260df662f18821a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:08:52.887542 7328 consensus_queue.cc:1035] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.888365 7205 raft_consensus.cc:1261] T 5c307d6bcbc84de79f0933a2b3b4d8cd P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Refusing update from remote peer af33d7febb294622a260df662f18821a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:08:52.888785 7278 consensus_queue.cc:1035] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.889269 7075 raft_consensus.cc:1261] T 70df2f4c2e024234abf62fe8fea726f7 P af33d7febb294622a260df662f18821a [term 1 FOLLOWER]: Refusing update from remote peer 60b660863ceb4187bcb3e8e45f7a5594: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:08:52.889873 7311 consensus_queue.cc:1035] T cc39e68201ff4ac2894f274498f92a4c P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.890141 7311 consensus_queue.cc:1035] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.892220 7320 consensus_queue.cc:1035] T 5c307d6bcbc84de79f0933a2b3b4d8cd P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.893733 7311 consensus_queue.cc:1035] T 70df2f4c2e024234abf62fe8fea726f7 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.898305 7335 consensus_queue.cc:1035] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.902312 6944 raft_consensus.cc:1261] T 06cac4dab02048839d0f5f0b989eae9c P 9c07440c3b8d4a939f56e878ef954d5e [term 1 FOLLOWER]: Refusing update from remote peer af33d7febb294622a260df662f18821a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:08:52.902662 7281 consensus_queue.cc:1035] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.904338 7204 raft_consensus.cc:1261] T 06cac4dab02048839d0f5f0b989eae9c P 60b660863ceb4187bcb3e8e45f7a5594 [term 1 FOLLOWER]: Refusing update from remote peer af33d7febb294622a260df662f18821a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:08:52.905035 7320 consensus_queue.cc:1035] T 06cac4dab02048839d0f5f0b989eae9c P af33d7febb294622a260df662f18821a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.919790 7403 mvcc.cc:205] Tried to move back new op lower bound from 7022889094697279488 to 7022889092984930304. Current Snapshot: MvccSnapshot[applied={T|T < 7022889094697279488}]
I20240501 14:08:52.925084 7276 consensus_queue.cc:1035] T 310fbceabf1f40fe8bd8b72e17e99bd4 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:52.931715 7358 mvcc.cc:205] Tried to move back new op lower bound from 7022889094699081728 to 7022889093634269184. Current Snapshot: MvccSnapshot[applied={T|T < 7022889094699081728}]
I20240501 14:08:52.939481 7418 mvcc.cc:205] Tried to move back new op lower bound from 7022889094691409920 to 7022889093254172672. Current Snapshot: MvccSnapshot[applied={T|T < 7022889094691409920}]
INSERT report
rows total: 2000
time total: 148.627 ms
time per row: 0.0743135 ms
I20240501 14:08:53.012720 7311 consensus_queue.cc:1035] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:53.016229 7278 consensus_queue.cc:1035] T ff02acd1ff0b47c3aa559c4ff4991df6 P 60b660863ceb4187bcb3e8e45f7a5594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c07440c3b8d4a939f56e878ef954d5e" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:53.030550 7276 consensus_queue.cc:1035] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:53.033779 7276 consensus_queue.cc:1035] T ed0c931c1560432c8a0ff141bb5adacd P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:53.058257 7336 consensus_queue.cc:1035] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b660863ceb4187bcb3e8e45f7a5594" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:53.062017 7276 consensus_queue.cc:1035] T 0980e052557a46cf91cb652c15eef515 P 9c07440c3b8d4a939f56e878ef954d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af33d7febb294622a260df662f18821a" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:53.190248 6918 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:08:53.190925 7179 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:08:53.193331 7046 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:08:53.291000 6918 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:08:53.291883 7179 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:08:53.292811 7046 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:08:53.414427 6918 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:08:53.416519 7179 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:08:53.417239 7046 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:08:53.652549 7046 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:08:53.652552 6918 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:08:53.654603 7179 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:08:53.747925 7046 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:08:53.748221 6918 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:08:53.748457 7179 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:08:53.782476 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 6862
I20240501 14:08:53.788338 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 6993
I20240501 14:08:53.793728 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 7124
I20240501 14:08:53.798918 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 6771
2024-05-01T14:08:53Z chronyd exiting
[ OK ] TableRangeRebalancingTest.Basic (6180 ms)
[----------] 1 test from TableRangeRebalancingTest (6180 ms total)
[----------] 1 test from RebalanceStartSafetyTest
[ RUN ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1
I20240501 14:08:53.825716 6759 test_util.cc:268] Using random seed: 114426029
I20240501 14:08:53.826488 6759 ts_itest-base.cc:112] Starting cluster with:
I20240501 14:08:53.826529 6759 ts_itest-base.cc:113] --------------
I20240501 14:08:53.826539 6759 ts_itest-base.cc:114] 5 tablet servers
I20240501 14:08:53.826548 6759 ts_itest-base.cc:115] 3 replicas per TS
I20240501 14:08:53.826557 6759 ts_itest-base.cc:116] --------------
2024-05-01T14:08:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T14:08:53Z Disabled control of system clock
I20240501 14:08:53.834064 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:43895
--webserver_interface=127.6.153.254
--webserver_port=0
--builtin_ntp_servers=127.6.153.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.153.254:43895
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 14:08:53.939301 7598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:53.939568 7598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:53.939625 7598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:53.942605 7598 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:53.942677 7598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 14:08:53.942703 7598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:53.942734 7598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 14:08:53.942763 7598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 14:08:53.946533 7598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.6.153.254:43895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:43895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.153.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:53.947607 7598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:53.948724 7598 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:53.955799 7606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:53.956075 7603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:53.956460 7598 server_base.cc:1034] running on GCE node
W20240501 14:08:53.956737 7604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:53.957233 7598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:53.958163 7598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:53.959321 7598 hybrid_clock.cc:648] HybridClock initialized: now 1714572533959307 us; error 29 us; skew 500 ppm
I20240501 14:08:53.961234 7598 webserver.cc:460] Webserver started at http://127.6.153.254:43833/ using document root <none> and password file <none>
I20240501 14:08:53.961766 7598 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:53.961819 7598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:53.962002 7598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:53.963722 7598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "993bbd5850b749f9aa1d3468386cd834"
format_stamp: "Formatted at 2024-05-01 14:08:53 on dist-test-slave-cmf0"
I20240501 14:08:53.964200 7598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "993bbd5850b749f9aa1d3468386cd834"
format_stamp: "Formatted at 2024-05-01 14:08:53 on dist-test-slave-cmf0"
I20240501 14:08:53.967444 7598 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:08:53.969563 7612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:53.970535 7598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:53.970646 7598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
uuid: "993bbd5850b749f9aa1d3468386cd834"
format_stamp: "Formatted at 2024-05-01 14:08:53 on dist-test-slave-cmf0"
I20240501 14:08:53.970731 7598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:53.983695 7598 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:53.984278 7598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:53.984427 7598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:53.990653 7598 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.254:43895
I20240501 14:08:53.990664 7664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.254:43895 every 8 connection(s)
I20240501 14:08:53.991590 7598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 14:08:53.993981 7665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:53.998737 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 7598
I20240501 14:08:53.998734 7665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834: Bootstrap starting.
I20240501 14:08:53.998839 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 14:08:54.001138 7665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:54.001808 7665 log.cc:824] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:54.003443 7665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834: No bootstrap required, opened a new log
I20240501 14:08:54.006064 7665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993bbd5850b749f9aa1d3468386cd834" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43895 } }
I20240501 14:08:54.006290 7665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:54.006350 7665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993bbd5850b749f9aa1d3468386cd834, State: Initialized, Role: FOLLOWER
I20240501 14:08:54.006784 7665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993bbd5850b749f9aa1d3468386cd834" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43895 } }
I20240501 14:08:54.006919 7665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:08:54.006978 7665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:08:54.007078 7665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:54.007910 7665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993bbd5850b749f9aa1d3468386cd834" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43895 } }
I20240501 14:08:54.008230 7665 leader_election.cc:302] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993bbd5850b749f9aa1d3468386cd834; no voters:
I20240501 14:08:54.008498 7665 leader_election.cc:288] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:08:54.008594 7670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:54.008798 7670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [term 1 LEADER]: Becoming Leader. State: Replica: 993bbd5850b749f9aa1d3468386cd834, State: Running, Role: LEADER
I20240501 14:08:54.009086 7670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993bbd5850b749f9aa1d3468386cd834" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43895 } }
I20240501 14:08:54.009527 7665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 14:08:54.010658 7672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993bbd5850b749f9aa1d3468386cd834. Latest consensus state: current_term: 1 leader_uuid: "993bbd5850b749f9aa1d3468386cd834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993bbd5850b749f9aa1d3468386cd834" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43895 } } }
I20240501 14:08:54.010788 7672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [sys.catalog]: This master's current role is: LEADER
I20240501 14:08:54.011005 7671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "993bbd5850b749f9aa1d3468386cd834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993bbd5850b749f9aa1d3468386cd834" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 43895 } } }
I20240501 14:08:54.011098 7671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834 [sys.catalog]: This master's current role is: LEADER
I20240501 14:08:54.011579 7679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 14:08:54.014017 7679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 14:08:54.018569 7679 catalog_manager.cc:1347] Generated new cluster ID: 9102f53bd0bb4a1085ddee9a4eed979a
I20240501 14:08:54.018635 7679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 14:08:54.031558 7679 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 14:08:54.032478 7679 catalog_manager.cc:1504] Loading token signing keys...
I20240501 14:08:54.042130 7679 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 993bbd5850b749f9aa1d3468386cd834: Generated new TSK 0
I20240501 14:08:54.042824 7679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 14:08:54.047863 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.193:0
--local_ip_for_outbound_sockets=127.6.153.193
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--builtin_ntp_servers=127.6.153.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:08:54.154948 7689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:54.155182 7689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:54.155234 7689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:54.158394 7689 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:54.158478 7689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:54.158553 7689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.193
I20240501 14:08:54.162371 7689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--never_fsync=true
--heap_profile_path=/tmp/kudu.7689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.193
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:54.163367 7689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:54.164520 7689 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:54.172324 7697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.172652 7695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.172958 7694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:54.173389 7689 server_base.cc:1034] running on GCE node
I20240501 14:08:54.173800 7689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:54.174444 7689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:54.175609 7689 hybrid_clock.cc:648] HybridClock initialized: now 1714572534175576 us; error 49 us; skew 500 ppm
I20240501 14:08:54.177615 7689 webserver.cc:460] Webserver started at http://127.6.153.193:46241/ using document root <none> and password file <none>
I20240501 14:08:54.178156 7689 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:54.178212 7689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:54.178400 7689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:54.180157 7689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "7b941663389f4c309e5f116786b84c9a"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.180648 7689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "7b941663389f4c309e5f116786b84c9a"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.184038 7689 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:08:54.186329 7703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.187366 7689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:08:54.187479 7689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "7b941663389f4c309e5f116786b84c9a"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.187572 7689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:54.204854 7689 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:54.205536 7689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:54.205701 7689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:54.206243 7689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:54.207118 7689 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:54.207171 7689 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.207235 7689 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:54.207263 7689 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.215685 7689 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.193:45707
I20240501 14:08:54.215711 7816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.193:45707 every 8 connection(s)
I20240501 14:08:54.216634 7689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 14:08:54.223675 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 7689
I20240501 14:08:54.223788 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 14:08:54.225488 7817 heartbeater.cc:347] Connected to a master server at 127.6.153.254:43895
I20240501 14:08:54.225731 7817 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:54.225965 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.194:0
--local_ip_for_outbound_sockets=127.6.153.194
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--builtin_ntp_servers=127.6.153.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:54.226311 7817 heartbeater.cc:511] Master 127.6.153.254:43895 requested a full tablet report, sending...
I20240501 14:08:54.228173 7627 ts_manager.cc:194] Registered new tserver with Master: 7b941663389f4c309e5f116786b84c9a (127.6.153.193:45707)
I20240501 14:08:54.229398 7627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.193:60413
W20240501 14:08:54.336179 7820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:54.336411 7820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:54.336454 7820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:54.339534 7820 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:54.339596 7820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:54.339672 7820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.194
I20240501 14:08:54.343271 7820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--never_fsync=true
--heap_profile_path=/tmp/kudu.7820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.194
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:54.344183 7820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:54.345278 7820 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:54.353459 7825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.353775 7828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.353900 7826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:54.354323 7820 server_base.cc:1034] running on GCE node
I20240501 14:08:54.354753 7820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:54.355306 7820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:54.356492 7820 hybrid_clock.cc:648] HybridClock initialized: now 1714572534356466 us; error 43 us; skew 500 ppm
I20240501 14:08:54.358472 7820 webserver.cc:460] Webserver started at http://127.6.153.194:44305/ using document root <none> and password file <none>
I20240501 14:08:54.359031 7820 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:54.359092 7820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:54.359274 7820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:54.360936 7820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "4a64904341f3459b88e383ffd3dc94c7"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.361402 7820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "4a64904341f3459b88e383ffd3dc94c7"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.364711 7820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:08:54.366859 7834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.367874 7820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:08:54.367980 7820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "4a64904341f3459b88e383ffd3dc94c7"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.368062 7820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:54.383015 7820 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:54.383689 7820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:54.383850 7820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:54.384363 7820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:54.385231 7820 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:54.385277 7820 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.385327 7820 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:54.385355 7820 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.393599 7820 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.194:44001
I20240501 14:08:54.393620 7947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.194:44001 every 8 connection(s)
I20240501 14:08:54.394578 7820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 14:08:54.401741 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 7820
I20240501 14:08:54.401858 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 14:08:54.403633 7948 heartbeater.cc:347] Connected to a master server at 127.6.153.254:43895
I20240501 14:08:54.403877 7948 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:54.404449 7948 heartbeater.cc:511] Master 127.6.153.254:43895 requested a full tablet report, sending...
I20240501 14:08:54.405460 7629 ts_manager.cc:194] Registered new tserver with Master: 4a64904341f3459b88e383ffd3dc94c7 (127.6.153.194:44001)
I20240501 14:08:54.406194 7629 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.194:36797
I20240501 14:08:54.406304 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:0
--local_ip_for_outbound_sockets=127.6.153.195
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--builtin_ntp_servers=127.6.153.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:08:54.511446 7951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:54.511693 7951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:54.511752 7951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:54.514806 7951 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:54.514874 7951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:54.514952 7951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:08:54.518611 7951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--never_fsync=true
--heap_profile_path=/tmp/kudu.7951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:54.519574 7951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:54.520699 7951 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:54.528946 7959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.529204 7956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.529601 7957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:54.530018 7951 server_base.cc:1034] running on GCE node
I20240501 14:08:54.530412 7951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:54.531028 7951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:54.532184 7951 hybrid_clock.cc:648] HybridClock initialized: now 1714572534532160 us; error 37 us; skew 500 ppm
I20240501 14:08:54.534094 7951 webserver.cc:460] Webserver started at http://127.6.153.195:37969/ using document root <none> and password file <none>
I20240501 14:08:54.534648 7951 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:54.534714 7951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:54.534896 7951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:54.536623 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "74963f177e354dd38b21454cfd62945d"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.537084 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "74963f177e354dd38b21454cfd62945d"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.540666 7951 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:08:54.542852 7965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.543881 7951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:08:54.543984 7951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "74963f177e354dd38b21454cfd62945d"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.544067 7951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:54.567517 7951 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:54.568104 7951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:54.568260 7951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:54.568786 7951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:54.569669 7951 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:54.569720 7951 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.569789 7951 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:54.569816 7951 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.577706 7951 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:34399
I20240501 14:08:54.577755 8078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:34399 every 8 connection(s)
I20240501 14:08:54.578648 7951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:08:54.581831 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 7951
I20240501 14:08:54.581923 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 14:08:54.584053 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.196:0
--local_ip_for_outbound_sockets=127.6.153.196
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--builtin_ntp_servers=127.6.153.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:54.587219 8079 heartbeater.cc:347] Connected to a master server at 127.6.153.254:43895
I20240501 14:08:54.587452 8079 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:54.587952 8079 heartbeater.cc:511] Master 127.6.153.254:43895 requested a full tablet report, sending...
I20240501 14:08:54.589699 7629 ts_manager.cc:194] Registered new tserver with Master: 74963f177e354dd38b21454cfd62945d (127.6.153.195:34399)
I20240501 14:08:54.590262 7629 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:54345
W20240501 14:08:54.689291 8082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:54.689489 8082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:54.689530 8082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:54.692576 8082 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:54.692652 8082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:54.692726 8082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.196
I20240501 14:08:54.696297 8082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--never_fsync=true
--heap_profile_path=/tmp/kudu.8082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.196
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:54.697137 8082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:54.698210 8082 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:54.706122 8088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.706441 8087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.706708 8090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:54.707051 8082 server_base.cc:1034] running on GCE node
I20240501 14:08:54.707430 8082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:54.707973 8082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:54.709143 8082 hybrid_clock.cc:648] HybridClock initialized: now 1714572534709111 us; error 44 us; skew 500 ppm
I20240501 14:08:54.711125 8082 webserver.cc:460] Webserver started at http://127.6.153.196:45439/ using document root <none> and password file <none>
I20240501 14:08:54.711828 8082 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:54.711901 8082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:54.712081 8082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:54.713687 8082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "883d77ebb4aa4035b65067655fb54522"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.714154 8082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "883d77ebb4aa4035b65067655fb54522"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.717314 8082 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:08:54.719506 8096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.720480 8082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:54.720595 8082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "883d77ebb4aa4035b65067655fb54522"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.720676 8082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:54.743146 8082 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:54.743811 8082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:54.743968 8082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:54.744485 8082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:54.745342 8082 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:54.745388 8082 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.745440 8082 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:54.745466 8082 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.753703 8082 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.196:40655
I20240501 14:08:54.753738 8209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.196:40655 every 8 connection(s)
I20240501 14:08:54.754657 8082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 14:08:54.759578 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 8082
I20240501 14:08:54.759665 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 14:08:54.761719 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.197:0
--local_ip_for_outbound_sockets=127.6.153.197
--webserver_interface=127.6.153.197
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--builtin_ntp_servers=127.6.153.212:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:54.763415 8210 heartbeater.cc:347] Connected to a master server at 127.6.153.254:43895
I20240501 14:08:54.763629 8210 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:54.764771 8210 heartbeater.cc:511] Master 127.6.153.254:43895 requested a full tablet report, sending...
I20240501 14:08:54.765744 7629 ts_manager.cc:194] Registered new tserver with Master: 883d77ebb4aa4035b65067655fb54522 (127.6.153.196:40655)
I20240501 14:08:54.766425 7629 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.196:45379
W20240501 14:08:54.870101 8213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:54.870314 8213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:54.870357 8213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:54.873330 8213 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:54.873392 8213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:54.873456 8213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.197
I20240501 14:08:54.876945 8213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.6.153.197
--webserver_port=0
--tserver_master_addrs=127.6.153.254:43895
--never_fsync=true
--heap_profile_path=/tmp/kudu.8213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.197
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:54.877735 8213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:54.878785 8213 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:54.886780 8219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.887100 8221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:54.887452 8218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:54.887897 8213 server_base.cc:1034] running on GCE node
I20240501 14:08:54.888223 8213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:54.888710 8213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:54.889844 8213 hybrid_clock.cc:648] HybridClock initialized: now 1714572534889820 us; error 38 us; skew 500 ppm
I20240501 14:08:54.891655 8213 webserver.cc:460] Webserver started at http://127.6.153.197:46095/ using document root <none> and password file <none>
I20240501 14:08:54.892144 8213 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:54.892196 8213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:54.892357 8213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:54.893931 8213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "f5604daa653247a4ad75f33c92cd4e10"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.894368 8213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "f5604daa653247a4ad75f33c92cd4e10"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.897575 8213 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:08:54.899636 8227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.900607 8213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 14:08:54.900714 8213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "f5604daa653247a4ad75f33c92cd4e10"
format_stamp: "Formatted at 2024-05-01 14:08:54 on dist-test-slave-cmf0"
I20240501 14:08:54.900797 8213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:54.915534 8213 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:54.916091 8213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:54.916242 8213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:54.916733 8213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:54.917578 8213 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:54.917624 8213 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.917675 8213 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:54.917703 8213 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:54.925558 8213 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.197:38703
I20240501 14:08:54.925593 8340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.197:38703 every 8 connection(s)
I20240501 14:08:54.926503 8213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 14:08:54.929057 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 8213
I20240501 14:08:54.929162 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 14:08:54.935515 8341 heartbeater.cc:347] Connected to a master server at 127.6.153.254:43895
I20240501 14:08:54.935721 8341 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:54.936221 8341 heartbeater.cc:511] Master 127.6.153.254:43895 requested a full tablet report, sending...
I20240501 14:08:54.937116 7629 ts_manager.cc:194] Registered new tserver with Master: f5604daa653247a4ad75f33c92cd4e10 (127.6.153.197:38703)
I20240501 14:08:54.937659 7629 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.197:43655
I20240501 14:08:54.943193 6759 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240501 14:08:54.951808 7629 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55112:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 14:08:54.968920 8012 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed703de9f86490695b2ee8fe7bc6649 (DEFAULT_TABLE table=TestTable [id=7f85d976539b4b0f8ba0c9bace36e2fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:08:54.968927 7882 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed703de9f86490695b2ee8fe7bc6649 (DEFAULT_TABLE table=TestTable [id=7f85d976539b4b0f8ba0c9bace36e2fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:08:54.969223 7751 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed703de9f86490695b2ee8fe7bc6649 (DEFAULT_TABLE table=TestTable [id=7f85d976539b4b0f8ba0c9bace36e2fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:08:54.969789 8012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed703de9f86490695b2ee8fe7bc6649. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:54.969826 7882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed703de9f86490695b2ee8fe7bc6649. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:54.969939 7751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed703de9f86490695b2ee8fe7bc6649. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:54.975664 8360 tablet_bootstrap.cc:492] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a: Bootstrap starting.
I20240501 14:08:54.975925 8358 tablet_bootstrap.cc:492] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d: Bootstrap starting.
I20240501 14:08:54.977511 8360 tablet_bootstrap.cc:654] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:54.977512 8358 tablet_bootstrap.cc:654] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:54.978159 8360 log.cc:824] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:54.978160 8358 log.cc:824] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:54.978813 8359 tablet_bootstrap.cc:492] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7: Bootstrap starting.
I20240501 14:08:54.979678 8358 tablet_bootstrap.cc:492] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d: No bootstrap required, opened a new log
I20240501 14:08:54.979843 8358 ts_tablet_manager.cc:1367] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240501 14:08:54.980340 8360 tablet_bootstrap.cc:492] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a: No bootstrap required, opened a new log
I20240501 14:08:54.980491 8360 ts_tablet_manager.cc:1367] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240501 14:08:54.980610 8359 tablet_bootstrap.cc:654] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:54.981298 8359 log.cc:824] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:54.982863 8359 tablet_bootstrap.cc:492] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7: No bootstrap required, opened a new log
I20240501 14:08:54.983031 8359 ts_tablet_manager.cc:1367] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 14:08:54.982926 8358 raft_consensus.cc:348] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:54.983119 8358 raft_consensus.cc:374] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:54.983219 8358 raft_consensus.cc:729] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74963f177e354dd38b21454cfd62945d, State: Initialized, Role: FOLLOWER
I20240501 14:08:54.983737 8358 consensus_queue.cc:260] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:54.983906 8360 raft_consensus.cc:348] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:54.984098 8360 raft_consensus.cc:374] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:54.984151 8360 raft_consensus.cc:729] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b941663389f4c309e5f116786b84c9a, State: Initialized, Role: FOLLOWER
I20240501 14:08:54.984567 8358 ts_tablet_manager.cc:1398] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 14:08:54.984560 8360 consensus_queue.cc:260] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:54.984901 8079 heartbeater.cc:503] Master 127.6.153.254:43895 was elected leader, sending a full tablet report...
I20240501 14:08:54.985323 8360 ts_tablet_manager.cc:1398] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 14:08:54.985664 7817 heartbeater.cc:503] Master 127.6.153.254:43895 was elected leader, sending a full tablet report...
I20240501 14:08:54.986440 8359 raft_consensus.cc:348] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:54.986629 8359 raft_consensus.cc:374] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:54.986692 8359 raft_consensus.cc:729] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a64904341f3459b88e383ffd3dc94c7, State: Initialized, Role: FOLLOWER
I20240501 14:08:54.987071 8359 consensus_queue.cc:260] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:54.987663 7948 heartbeater.cc:503] Master 127.6.153.254:43895 was elected leader, sending a full tablet report...
I20240501 14:08:54.987876 8359 ts_tablet_manager.cc:1398] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 14:08:54.991210 8366 raft_consensus.cc:482] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:54.991367 8366 raft_consensus.cc:504] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:54.992381 8366 leader_election.cc:288] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74963f177e354dd38b21454cfd62945d (127.6.153.195:34399), 7b941663389f4c309e5f116786b84c9a (127.6.153.193:45707)
I20240501 14:08:54.996470 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed703de9f86490695b2ee8fe7bc6649" candidate_uuid: "4a64904341f3459b88e383ffd3dc94c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b941663389f4c309e5f116786b84c9a" is_pre_election: true
I20240501 14:08:54.996796 7771 raft_consensus.cc:2454] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a64904341f3459b88e383ffd3dc94c7 in term 0.
I20240501 14:08:54.997038 8033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed703de9f86490695b2ee8fe7bc6649" candidate_uuid: "4a64904341f3459b88e383ffd3dc94c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74963f177e354dd38b21454cfd62945d" is_pre_election: true
I20240501 14:08:54.997257 7835 leader_election.cc:302] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a64904341f3459b88e383ffd3dc94c7, 7b941663389f4c309e5f116786b84c9a; no voters:
I20240501 14:08:54.997288 8033 raft_consensus.cc:2454] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a64904341f3459b88e383ffd3dc94c7 in term 0.
I20240501 14:08:54.997476 8366 raft_consensus.cc:2781] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:54.997550 8366 raft_consensus.cc:482] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:54.997589 8366 raft_consensus.cc:3037] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:55.004952 8366 raft_consensus.cc:504] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:55.005445 8366 leader_election.cc:288] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [CANDIDATE]: Term 1 election: Requested vote from peers 74963f177e354dd38b21454cfd62945d (127.6.153.195:34399), 7b941663389f4c309e5f116786b84c9a (127.6.153.193:45707)
I20240501 14:08:55.005890 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed703de9f86490695b2ee8fe7bc6649" candidate_uuid: "4a64904341f3459b88e383ffd3dc94c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b941663389f4c309e5f116786b84c9a"
I20240501 14:08:55.005923 8033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed703de9f86490695b2ee8fe7bc6649" candidate_uuid: "4a64904341f3459b88e383ffd3dc94c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74963f177e354dd38b21454cfd62945d"
I20240501 14:08:55.006032 7771 raft_consensus.cc:3037] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:55.006045 8033 raft_consensus.cc:3037] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:55.007048 7771 raft_consensus.cc:2454] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a64904341f3459b88e383ffd3dc94c7 in term 1.
I20240501 14:08:55.007140 8033 raft_consensus.cc:2454] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a64904341f3459b88e383ffd3dc94c7 in term 1.
I20240501 14:08:55.007510 7838 leader_election.cc:302] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a64904341f3459b88e383ffd3dc94c7, 74963f177e354dd38b21454cfd62945d; no voters:
I20240501 14:08:55.007684 8366 raft_consensus.cc:2781] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:55.007941 8366 raft_consensus.cc:686] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [term 1 LEADER]: Becoming Leader. State: Replica: 4a64904341f3459b88e383ffd3dc94c7, State: Running, Role: LEADER
I20240501 14:08:55.008298 8366 consensus_queue.cc:237] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } }
I20240501 14:08:55.011755 7628 catalog_manager.cc:5484] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a64904341f3459b88e383ffd3dc94c7 (127.6.153.194). New cstate: current_term: 1 leader_uuid: "4a64904341f3459b88e383ffd3dc94c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a64904341f3459b88e383ffd3dc94c7" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 44001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:55.030109 6759 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240501 14:08:55.031764 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 7b941663389f4c309e5f116786b84c9a to finish bootstrapping
I20240501 14:08:55.037406 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 4a64904341f3459b88e383ffd3dc94c7 to finish bootstrapping
I20240501 14:08:55.042047 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 74963f177e354dd38b21454cfd62945d to finish bootstrapping
I20240501 14:08:55.047015 6759 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 883d77ebb4aa4035b65067655fb54522 to finish bootstrapping
I20240501 14:08:55.053110 6759 ts_itest-base.cc:233] Waiting for 0 tablets on tserver f5604daa653247a4ad75f33c92cd4e10 to finish bootstrapping
W20240501 14:08:55.079648 8080 tablet.cc:2369] T 8ed703de9f86490695b2ee8fe7bc6649 P 74963f177e354dd38b21454cfd62945d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:08:55.145789 7949 tablet.cc:2369] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:08:55.218052 7818 tablet.cc:2369] T 8ed703de9f86490695b2ee8fe7bc6649 P 7b941663389f4c309e5f116786b84c9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:08:55.242259 7882 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:08:55.242374 8012 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.243317 7751 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.244036 8144 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.244444 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.340090 7882 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:08:55.340845 8012 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.341096 8144 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.342399 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.348938 7751 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.451979 8012 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.452731 7751 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.453394 7882 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:08:55.454787 8144 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.458791 8366 consensus_queue.cc:1035] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b941663389f4c309e5f116786b84c9a" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:55.460675 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.464195 8366 consensus_queue.cc:1035] T 8ed703de9f86490695b2ee8fe7bc6649 P 4a64904341f3459b88e383ffd3dc94c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74963f177e354dd38b21454cfd62945d" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 34399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:55.688227 8012 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.688324 7882 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:08:55.689790 8144 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.690191 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.690519 7751 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.768384 8210 heartbeater.cc:503] Master 127.6.153.254:43895 was elected leader, sending a full tablet report...
I20240501 14:08:55.781483 8012 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.781517 7882 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:08:55.782584 7751 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.783133 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.783242 8144 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.883136 7882 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:08:55.884426 8144 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.884696 7751 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.885056 8012 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.887890 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:08:55.927374 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 7689
I20240501 14:08:55.935168 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 7820
I20240501 14:08:55.939508 8341 heartbeater.cc:503] Master 127.6.153.254:43895 was elected leader, sending a full tablet report...
I20240501 14:08:55.940215 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 7951
I20240501 14:08:55.945091 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 8082
I20240501 14:08:55.949604 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 8213
I20240501 14:08:55.954545 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 7598
2024-05-01T14:08:55Z chronyd exiting
[ OK ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1 (2149 ms)
[----------] 1 test from RebalanceStartSafetyTest (2149 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/7
I20240501 14:08:55.975811 6759 test_util.cc:268] Using random seed: 116576125
I20240501 14:08:55.976615 6759 ts_itest-base.cc:112] Starting cluster with:
I20240501 14:08:55.976665 6759 ts_itest-base.cc:113] --------------
I20240501 14:08:55.976675 6759 ts_itest-base.cc:114] 7 tablet servers
I20240501 14:08:55.976682 6759 ts_itest-base.cc:115] 5 replicas per TS
I20240501 14:08:55.976691 6759 ts_itest-base.cc:116] --------------
2024-05-01T14:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T14:08:55Z Disabled control of system clock
I20240501 14:08:55.984174 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:42171
--webserver_interface=127.6.153.254
--webserver_port=0
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.153.254:42171
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240501 14:08:56.095304 8581 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 14:08:56.095600 8581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:56.095633 8581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:56.095676 8581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:56.098752 8581 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:56.098827 8581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 14:08:56.098850 8581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:56.098883 8581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 14:08:56.098917 8581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 14:08:56.102670 8581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.6.153.254:42171
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:42171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.153.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.8581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:56.103638 8581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:56.104761 8581 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:56.111835 8586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.112259 8587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.112524 8589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:56.112932 8581 server_base.cc:1034] running on GCE node
I20240501 14:08:56.113375 8581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:56.114246 8581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:56.115384 8581 hybrid_clock.cc:648] HybridClock initialized: now 1714572536115362 us; error 36 us; skew 500 ppm
I20240501 14:08:56.117213 8581 webserver.cc:460] Webserver started at http://127.6.153.254:42511/ using document root <none> and password file <none>
I20240501 14:08:56.117769 8581 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:56.117823 8581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:56.118006 8581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:56.119678 8581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "9fa87eb4295445af9eb74cc9bbb89994"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.120142 8581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "9fa87eb4295445af9eb74cc9bbb89994"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.123286 8581 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:08:56.125407 8595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.126344 8581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 14:08:56.126451 8581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
uuid: "9fa87eb4295445af9eb74cc9bbb89994"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.126533 8581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:56.141505 8581 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:56.142148 8581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:56.142307 8581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:56.148805 8581 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.254:42171
I20240501 14:08:56.148820 8647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.254:42171 every 8 connection(s)
I20240501 14:08:56.149778 8581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 14:08:56.152273 8648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:56.157213 8648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994: Bootstrap starting.
I20240501 14:08:56.159457 8648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:56.159844 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 8581
I20240501 14:08:56.159937 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 14:08:56.160171 8648 log.cc:824] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:56.162084 8648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994: No bootstrap required, opened a new log
I20240501 14:08:56.165377 8648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa87eb4295445af9eb74cc9bbb89994" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 42171 } }
I20240501 14:08:56.165558 8648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:56.165594 8648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fa87eb4295445af9eb74cc9bbb89994, State: Initialized, Role: FOLLOWER
I20240501 14:08:56.166030 8648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa87eb4295445af9eb74cc9bbb89994" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 42171 } }
I20240501 14:08:56.166178 8648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:08:56.166241 8648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:08:56.166349 8648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:56.180220 8648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa87eb4295445af9eb74cc9bbb89994" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 42171 } }
I20240501 14:08:56.180662 8648 leader_election.cc:302] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fa87eb4295445af9eb74cc9bbb89994; no voters:
I20240501 14:08:56.180981 8648 leader_election.cc:288] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:08:56.181389 8653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:56.181588 8653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [term 1 LEADER]: Becoming Leader. State: Replica: 9fa87eb4295445af9eb74cc9bbb89994, State: Running, Role: LEADER
I20240501 14:08:56.181907 8653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa87eb4295445af9eb74cc9bbb89994" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 42171 } }
I20240501 14:08:56.182215 8648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 14:08:56.183482 8655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fa87eb4295445af9eb74cc9bbb89994. Latest consensus state: current_term: 1 leader_uuid: "9fa87eb4295445af9eb74cc9bbb89994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa87eb4295445af9eb74cc9bbb89994" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 42171 } } }
I20240501 14:08:56.183609 8655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [sys.catalog]: This master's current role is: LEADER
I20240501 14:08:56.183907 8654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fa87eb4295445af9eb74cc9bbb89994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa87eb4295445af9eb74cc9bbb89994" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 42171 } } }
I20240501 14:08:56.184013 8654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994 [sys.catalog]: This master's current role is: LEADER
I20240501 14:08:56.184181 8662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 14:08:56.187073 8662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 14:08:56.192198 8662 catalog_manager.cc:1347] Generated new cluster ID: 306a3d7b9e3842ad9bc7b8b1cee40268
I20240501 14:08:56.192263 8662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 14:08:56.203127 8662 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 14:08:56.204401 8662 catalog_manager.cc:1504] Loading token signing keys...
I20240501 14:08:56.213418 8662 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 9fa87eb4295445af9eb74cc9bbb89994: Generated new TSK 0
I20240501 14:08:56.214289 8662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 14:08:56.228353 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.193:0
--local_ip_for_outbound_sockets=127.6.153.193
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:08:56.336565 8672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:56.336776 8672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:56.336836 8672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:56.339926 8672 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:56.340001 8672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:56.340087 8672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.193
I20240501 14:08:56.343802 8672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.8672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.193
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:56.344666 8672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:56.345733 8672 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:56.353606 8678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.353852 8680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.354144 8677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:56.354663 8672 server_base.cc:1034] running on GCE node
I20240501 14:08:56.355011 8672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:56.355599 8672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:56.356741 8672 hybrid_clock.cc:648] HybridClock initialized: now 1714572536356722 us; error 33 us; skew 500 ppm
I20240501 14:08:56.358587 8672 webserver.cc:460] Webserver started at http://127.6.153.193:35865/ using document root <none> and password file <none>
I20240501 14:08:56.359119 8672 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:56.359184 8672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:56.359398 8672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:56.361021 8672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "25737230978048598f426b1a99b27441"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.361490 8672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "25737230978048598f426b1a99b27441"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.364671 8672 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240501 14:08:56.366668 8686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.367646 8672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:56.367763 8672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "25737230978048598f426b1a99b27441"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.367843 8672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:56.396538 8672 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:56.397187 8672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:56.397354 8672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:56.397878 8672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:56.398777 8672 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:56.398823 8672 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.398873 8672 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:56.398901 8672 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.407562 8672 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.193:41195
I20240501 14:08:56.407584 8799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.193:41195 every 8 connection(s)
I20240501 14:08:56.408521 8672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 14:08:56.414567 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 8672
I20240501 14:08:56.414691 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 14:08:56.416808 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.194:0
--local_ip_for_outbound_sockets=127.6.153.194
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:56.417186 8800 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:08:56.417438 8800 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:56.418082 8800 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:08:56.419580 8612 ts_manager.cc:194] Registered new tserver with Master: 25737230978048598f426b1a99b27441 (127.6.153.193:41195)
I20240501 14:08:56.420754 8612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.193:59481
W20240501 14:08:56.523159 8803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:56.523440 8803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:56.523507 8803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:56.526631 8803 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:56.526707 8803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:56.526795 8803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.194
I20240501 14:08:56.530622 8803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.8803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.194
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:56.531800 8803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:56.533013 8803 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:56.541324 8809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.541601 8811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.542124 8808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:56.542441 8803 server_base.cc:1034] running on GCE node
I20240501 14:08:56.542804 8803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:56.543447 8803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:56.544603 8803 hybrid_clock.cc:648] HybridClock initialized: now 1714572536544589 us; error 29 us; skew 500 ppm
I20240501 14:08:56.546507 8803 webserver.cc:460] Webserver started at http://127.6.153.194:40741/ using document root <none> and password file <none>
I20240501 14:08:56.547057 8803 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:56.547116 8803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:56.547298 8803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:56.548936 8803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.549387 8803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.552778 8803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240501 14:08:56.554879 8817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.555907 8803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:56.556010 8803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.556090 8803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:56.577049 8803 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:56.577782 8803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:56.577950 8803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:56.578529 8803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:56.579505 8803 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:56.579564 8803 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.579631 8803 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:56.579660 8803 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.588313 8803 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.194:35221
I20240501 14:08:56.588340 8930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.194:35221 every 8 connection(s)
I20240501 14:08:56.589277 8803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 14:08:56.592710 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 8803
I20240501 14:08:56.592800 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 14:08:56.594815 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:0
--local_ip_for_outbound_sockets=127.6.153.195
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:56.599370 8931 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:08:56.599606 8931 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:56.600157 8931 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:08:56.601168 8612 ts_manager.cc:194] Registered new tserver with Master: f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:08:56.601733 8612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.194:50575
W20240501 14:08:56.706216 8934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:56.706465 8934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:56.706524 8934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:56.709831 8934 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:56.709906 8934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:56.710001 8934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:08:56.713877 8934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.8934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:56.714874 8934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:56.716228 8934 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:56.724457 8942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.724767 8940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.725203 8939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:56.725629 8934 server_base.cc:1034] running on GCE node
I20240501 14:08:56.725975 8934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:56.726531 8934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:56.727699 8934 hybrid_clock.cc:648] HybridClock initialized: now 1714572536727666 us; error 48 us; skew 500 ppm
I20240501 14:08:56.729704 8934 webserver.cc:460] Webserver started at http://127.6.153.195:32859/ using document root <none> and password file <none>
I20240501 14:08:56.730268 8934 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:56.730337 8934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:56.730540 8934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:56.732290 8934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "84c7d91c82524b69a9184fcf1949b5b0"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.732760 8934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "84c7d91c82524b69a9184fcf1949b5b0"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.736064 8934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:08:56.738297 8948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.739503 8934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:08:56.739611 8934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "84c7d91c82524b69a9184fcf1949b5b0"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.739706 8934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:56.759544 8934 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:56.760227 8934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:56.760391 8934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:56.761004 8934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:56.761936 8934 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:56.761986 8934 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.762039 8934 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:56.762069 8934 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.770536 8934 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:33719
I20240501 14:08:56.770574 9061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:33719 every 8 connection(s)
I20240501 14:08:56.771610 8934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:08:56.780820 9062 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:08:56.781052 9062 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:56.781200 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 8934
I20240501 14:08:56.781288 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 14:08:56.781812 9062 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:08:56.782882 8612 ts_manager.cc:194] Registered new tserver with Master: 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719)
I20240501 14:08:56.783308 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.196:0
--local_ip_for_outbound_sockets=127.6.153.196
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:56.783600 8612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:59923
W20240501 14:08:56.902102 9065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:56.902333 9065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:56.902379 9065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:56.905548 9065 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:56.905635 9065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:56.905712 9065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.196
I20240501 14:08:56.909785 9065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.9065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.196
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:56.910771 9065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:56.911957 9065 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:56.920662 9071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.920950 9070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:56.921418 9073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:56.921903 9065 server_base.cc:1034] running on GCE node
I20240501 14:08:56.922329 9065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:56.922917 9065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:56.924088 9065 hybrid_clock.cc:648] HybridClock initialized: now 1714572536924060 us; error 46 us; skew 500 ppm
I20240501 14:08:56.926036 9065 webserver.cc:460] Webserver started at http://127.6.153.196:44373/ using document root <none> and password file <none>
I20240501 14:08:56.926635 9065 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:56.926693 9065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:56.926884 9065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:56.928661 9065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.929148 9065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.932766 9065 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:08:56.935182 9079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.936269 9065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:08:56.936375 9065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
format_stamp: "Formatted at 2024-05-01 14:08:56 on dist-test-slave-cmf0"
I20240501 14:08:56.936458 9065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:56.949244 9065 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:56.949856 9065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:56.950004 9065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:56.950544 9065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:56.951606 9065 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:56.951656 9065 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.951709 9065 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:56.951735 9065 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:56.959666 9065 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.196:34983
I20240501 14:08:56.959707 9192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.196:34983 every 8 connection(s)
I20240501 14:08:56.960636 9065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 14:08:56.969141 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 9065
I20240501 14:08:56.969228 9193 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:08:56.969266 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 14:08:56.969470 9193 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:56.970088 9193 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:08:56.971117 8612 ts_manager.cc:194] Registered new tserver with Master: 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983)
I20240501 14:08:56.971525 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.197:0
--local_ip_for_outbound_sockets=127.6.153.197
--webserver_interface=127.6.153.197
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:56.971845 8612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.196:38039
W20240501 14:08:57.077963 9196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:57.078217 9196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:57.078263 9196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:57.081382 9196 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:57.081451 9196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:57.081521 9196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.197
I20240501 14:08:57.085283 9196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.6.153.197
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.9196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.197
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:57.086144 9196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:57.087286 9196 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:57.095321 9202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:57.095597 9204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:57.095943 9201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:57.096447 9196 server_base.cc:1034] running on GCE node
I20240501 14:08:57.096850 9196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:57.097478 9196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:57.098659 9196 hybrid_clock.cc:648] HybridClock initialized: now 1714572537098642 us; error 31 us; skew 500 ppm
I20240501 14:08:57.100961 9196 webserver.cc:460] Webserver started at http://127.6.153.197:41771/ using document root <none> and password file <none>
I20240501 14:08:57.101547 9196 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:57.101612 9196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:57.101775 9196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:57.103498 9196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "0c0d3d3aae054876a0a522a8325ddd02"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.103952 9196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "0c0d3d3aae054876a0a522a8325ddd02"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.107290 9196 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:08:57.109604 9210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.110742 9196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 14:08:57.110853 9196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "0c0d3d3aae054876a0a522a8325ddd02"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.110956 9196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:57.146340 9196 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:57.147014 9196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:57.147156 9196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:57.147765 9196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:57.148686 9196 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:57.148733 9196 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.148785 9196 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:57.148813 9196 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.157824 9196 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.197:42941
I20240501 14:08:57.157897 9323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.197:42941 every 8 connection(s)
I20240501 14:08:57.158752 9196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 14:08:57.167438 9324 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:08:57.167649 9324 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:57.167867 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 9196
I20240501 14:08:57.167959 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 14:08:57.168226 9324 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:08:57.169171 8611 ts_manager.cc:194] Registered new tserver with Master: 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:08:57.169811 8611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.197:50393
I20240501 14:08:57.170089 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.198:0
--local_ip_for_outbound_sockets=127.6.153.198
--webserver_interface=127.6.153.198
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:08:57.276417 9327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:57.276665 9327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:57.276710 9327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:57.279793 9327 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:57.279855 9327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:57.279923 9327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.198
I20240501 14:08:57.283777 9327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.198:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.6.153.198
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.9327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.198
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:57.284729 9327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:57.285821 9327 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:57.293946 9335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:57.294236 9333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:57.294596 9332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:57.295020 9327 server_base.cc:1034] running on GCE node
I20240501 14:08:57.295408 9327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:57.296016 9327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:57.297179 9327 hybrid_clock.cc:648] HybridClock initialized: now 1714572537297165 us; error 34 us; skew 500 ppm
I20240501 14:08:57.299122 9327 webserver.cc:460] Webserver started at http://127.6.153.198:45463/ using document root <none> and password file <none>
I20240501 14:08:57.299705 9327 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:57.299764 9327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:57.299937 9327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:57.301607 9327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.302073 9327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.305739 9327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240501 14:08:57.308130 9341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.309211 9327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:08:57.309322 9327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.309424 9327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:57.323773 9327 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:57.324463 9327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:57.324617 9327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:57.325165 9327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:57.326082 9327 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:57.326135 9327 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.326175 9327 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:57.326189 9327 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.334784 9327 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.198:32925
I20240501 14:08:57.334806 9454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.198:32925 every 8 connection(s)
I20240501 14:08:57.335839 9327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 14:08:57.344871 9455 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:08:57.345162 9455 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:57.345731 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 9327
I20240501 14:08:57.345764 9455 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:08:57.345821 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 14:08:57.346832 8611 ts_manager.cc:194] Registered new tserver with Master: 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:08:57.347468 8611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.198:37973
I20240501 14:08:57.347925 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.199:0
--local_ip_for_outbound_sockets=127.6.153.199
--webserver_interface=127.6.153.199
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:08:57.423458 8800 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
W20240501 14:08:57.457155 9458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:08:57.457401 9458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:08:57.457460 9458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:08:57.460647 9458 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:08:57.460732 9458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:08:57.460822 9458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.199
I20240501 14:08:57.464687 9458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.199:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.6.153.199
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.9458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.199
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:08:57.465679 9458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:08:57.466869 9458 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:08:57.475437 9464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:57.475728 9466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:08:57.475978 9463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:08:57.476498 9458 server_base.cc:1034] running on GCE node
I20240501 14:08:57.476960 9458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:08:57.477551 9458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:08:57.478711 9458 hybrid_clock.cc:648] HybridClock initialized: now 1714572537478687 us; error 39 us; skew 500 ppm
I20240501 14:08:57.480763 9458 webserver.cc:460] Webserver started at http://127.6.153.199:43067/ using document root <none> and password file <none>
I20240501 14:08:57.481333 9458 fs_manager.cc:359] Metadata directory not provided
I20240501 14:08:57.481389 9458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:08:57.481577 9458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:08:57.483419 9458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "9a65600939334f6db8dcca7f07923ece"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.483916 9458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "9a65600939334f6db8dcca7f07923ece"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.487311 9458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240501 14:08:57.489632 9472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.490779 9458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 14:08:57.491513 9458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "9a65600939334f6db8dcca7f07923ece"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:08:57.491663 9458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:08:57.521459 9458 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:08:57.522109 9458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:08:57.522257 9458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:08:57.522776 9458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:08:57.523726 9458 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:08:57.523774 9458 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.523813 9458 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:08:57.523833 9458 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:08:57.531926 9458 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.199:42105
I20240501 14:08:57.531953 9585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.199:42105 every 8 connection(s)
I20240501 14:08:57.532850 9458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 14:08:57.533689 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 9458
I20240501 14:08:57.533773 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 14:08:57.541455 9586 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:08:57.541682 9586 heartbeater.cc:464] Registering TS with master...
I20240501 14:08:57.542201 9586 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:08:57.543166 8611 ts_manager.cc:194] Registered new tserver with Master: 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105)
I20240501 14:08:57.543756 8611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.199:56031
I20240501 14:08:57.547816 6759 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240501 14:08:57.556547 8611 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40168:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 14:08:57.576067 8996 tablet_service.cc:1467] Processing CreateTablet for tablet ab008693c1db49e493c3fac38646c8c0 (DEFAULT_TABLE table=TestTable [id=64c83efbafb94a168955877da1bf4dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:08:57.576081 9388 tablet_service.cc:1467] Processing CreateTablet for tablet ab008693c1db49e493c3fac38646c8c0 (DEFAULT_TABLE table=TestTable [id=64c83efbafb94a168955877da1bf4dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:08:57.577162 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab008693c1db49e493c3fac38646c8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:57.577162 9388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab008693c1db49e493c3fac38646c8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:57.579730 9520 tablet_service.cc:1467] Processing CreateTablet for tablet ab008693c1db49e493c3fac38646c8c0 (DEFAULT_TABLE table=TestTable [id=64c83efbafb94a168955877da1bf4dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:08:57.580577 9520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab008693c1db49e493c3fac38646c8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:57.581269 9248 tablet_service.cc:1467] Processing CreateTablet for tablet ab008693c1db49e493c3fac38646c8c0 (DEFAULT_TABLE table=TestTable [id=64c83efbafb94a168955877da1bf4dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:08:57.582214 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab008693c1db49e493c3fac38646c8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:57.583945 9608 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:08:57.586333 9608 tablet_bootstrap.cc:654] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:57.587663 9608 log.cc:824] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:57.588265 9609 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:08:57.590447 9609 tablet_bootstrap.cc:654] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:57.591223 9609 log.cc:824] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:57.591125 9127 tablet_service.cc:1467] Processing CreateTablet for tablet ab008693c1db49e493c3fac38646c8c0 (DEFAULT_TABLE table=TestTable [id=64c83efbafb94a168955877da1bf4dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:08:57.592015 9610 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:08:57.592130 9127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab008693c1db49e493c3fac38646c8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:08:57.593206 9612 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:08:57.593930 9608 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:08:57.594107 9610 tablet_bootstrap.cc:654] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:57.594108 9608 ts_tablet_manager.cc:1367] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20240501 14:08:57.594913 9610 log.cc:824] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:57.595258 9612 tablet_bootstrap.cc:654] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:57.596074 9612 log.cc:824] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:57.596693 9609 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:08:57.596875 9609 ts_tablet_manager.cc:1367] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20240501 14:08:57.597592 9612 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:08:57.597738 9612 ts_tablet_manager.cc:1367] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240501 14:08:57.598763 9616 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:08:57.598866 9608 raft_consensus.cc:348] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.599105 9608 raft_consensus.cc:374] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:57.599148 9608 raft_consensus.cc:729] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:08:57.599617 9608 consensus_queue.cc:260] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.600525 9609 raft_consensus.cc:348] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.600734 9609 raft_consensus.cc:374] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:57.600800 9609 raft_consensus.cc:729] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:08:57.600687 9612 raft_consensus.cc:348] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.600908 9612 raft_consensus.cc:374] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:57.600966 9612 raft_consensus.cc:729] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:08:57.601238 9609 consensus_queue.cc:260] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.601430 9616 tablet_bootstrap.cc:654] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:08:57.601387 9612 consensus_queue.cc:260] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.601447 9608 ts_tablet_manager.cc:1398] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240501 14:08:57.601902 9455 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
I20240501 14:08:57.602037 9609 ts_tablet_manager.cc:1398] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240501 14:08:57.602164 9612 ts_tablet_manager.cc:1398] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 14:08:57.602191 9616 log.cc:824] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: Log is configured to *not* fsync() on all Append() calls
I20240501 14:08:57.602473 9062 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
I20240501 14:08:57.602994 9324 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
I20240501 14:08:57.604265 9610 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: No bootstrap required, opened a new log
I20240501 14:08:57.604450 9610 ts_tablet_manager.cc:1367] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20240501 14:08:57.604624 9616 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:08:57.604764 9616 ts_tablet_manager.cc:1367] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240501 14:08:57.604951 8931 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
I20240501 14:08:57.607879 9616 raft_consensus.cc:348] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.608134 9616 raft_consensus.cc:374] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:57.608193 9616 raft_consensus.cc:729] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:08:57.608374 9610 raft_consensus.cc:348] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.608570 9610 raft_consensus.cc:374] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:08:57.608621 9610 raft_consensus.cc:729] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: FOLLOWER
I20240501 14:08:57.608630 9616 consensus_queue.cc:260] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.609043 9610 consensus_queue.cc:260] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.609391 9193 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
I20240501 14:08:57.609529 9616 ts_tablet_manager.cc:1398] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 14:08:57.609833 9610 ts_tablet_manager.cc:1398] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240501 14:08:57.609925 9586 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
W20240501 14:08:57.659991 9325 tablet.cc:2369] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:08:57.696487 9621 raft_consensus.cc:482] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:57.696720 9621 raft_consensus.cc:504] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.698385 9621 leader_election.cc:288] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105)
I20240501 14:08:57.703675 9617 raft_consensus.cc:482] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:08:57.703830 9540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a65600939334f6db8dcca7f07923ece" is_pre_election: true
I20240501 14:08:57.703830 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:08:57.703878 9617 raft_consensus.cc:504] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.703943 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:08:57.704128 9540 raft_consensus.cc:2454] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 0.
I20240501 14:08:57.704134 9278 raft_consensus.cc:2454] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 0.
I20240501 14:08:57.704195 9016 raft_consensus.cc:2454] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 0.
I20240501 14:08:57.704574 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:08:57.704756 9409 raft_consensus.cc:2454] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 0.
I20240501 14:08:57.704928 9081 leader_election.cc:302] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 73ea0b7a5e4c45a3814d46e74d62331a, 84c7d91c82524b69a9184fcf1949b5b0, 9a65600939334f6db8dcca7f07923ece; no voters:
I20240501 14:08:57.705195 9621 raft_consensus.cc:2781] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:08:57.705256 9621 raft_consensus.cc:482] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:08:57.705292 9621 raft_consensus.cc:3037] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:57.705315 9617 leader_election.cc:288] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105)
I20240501 14:08:57.706444 9621 raft_consensus.cc:504] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.706943 9621 leader_election.cc:288] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [CANDIDATE]: Term 1 election: Requested vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105)
I20240501 14:08:57.707729 9540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a65600939334f6db8dcca7f07923ece"
I20240501 14:08:57.707861 9540 raft_consensus.cc:3037] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:57.708974 9540 raft_consensus.cc:2454] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1.
I20240501 14:08:57.710263 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:08:57.710402 9409 raft_consensus.cc:3037] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:57.710692 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:08:57.710820 9016 raft_consensus.cc:3037] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:57.710945 9540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a65600939334f6db8dcca7f07923ece" is_pre_election: true
I20240501 14:08:57.711414 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:08:57.711552 9540 raft_consensus.cc:2379] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b961a167ae64dbc9d102ce5ddf721f3 in current term 1: Already voted for candidate 73ea0b7a5e4c45a3814d46e74d62331a in this term.
I20240501 14:08:57.711539 9278 raft_consensus.cc:3037] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:08:57.711551 9409 raft_consensus.cc:2454] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1.
I20240501 14:08:57.711822 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
W20240501 14:08:57.711998 9194 tablet.cc:2369] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:08:57.712138 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:08:57.712388 9147 raft_consensus.cc:2379] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b961a167ae64dbc9d102ce5ddf721f3 in current term 1: Already voted for candidate 73ea0b7a5e4c45a3814d46e74d62331a in this term.
I20240501 14:08:57.712611 9278 raft_consensus.cc:2454] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1.
I20240501 14:08:57.712687 9080 leader_election.cc:302] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6b961a167ae64dbc9d102ce5ddf721f3, 73ea0b7a5e4c45a3814d46e74d62331a, 9a65600939334f6db8dcca7f07923ece; no voters:
I20240501 14:08:57.712865 9621 raft_consensus.cc:2781] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:08:57.712983 9343 leader_election.cc:302] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 5 voters: 1 yes votes; 3 no votes. yes voters: 6b961a167ae64dbc9d102ce5ddf721f3; no voters: 0c0d3d3aae054876a0a522a8325ddd02, 73ea0b7a5e4c45a3814d46e74d62331a, 9a65600939334f6db8dcca7f07923ece
I20240501 14:08:57.713064 9621 raft_consensus.cc:686] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 LEADER]: Becoming Leader. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Running, Role: LEADER
I20240501 14:08:57.713248 9617 raft_consensus.cc:2726] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 14:08:57.713399 9621 consensus_queue.cc:237] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:08:57.714177 9016 raft_consensus.cc:2454] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1.
I20240501 14:08:57.714452 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab008693c1db49e493c3fac38646c8c0" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:08:57.714605 9015 raft_consensus.cc:2379] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b961a167ae64dbc9d102ce5ddf721f3 in current term 1: Already voted for candidate 73ea0b7a5e4c45a3814d46e74d62331a in this term.
I20240501 14:08:57.716907 8612 catalog_manager.cc:5484] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a reported cstate change: term changed from 0 to 1, leader changed from <none> to 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196). New cstate: current_term: 1 leader_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } health_report { overall_health: UNKNOWN } } }
I20240501 14:08:57.739117 6759 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240501 14:08:57.741147 6759 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 25737230978048598f426b1a99b27441 to finish bootstrapping
I20240501 14:08:57.747761 6759 ts_itest-base.cc:233] Waiting for 0 tablets on tserver f2e6587c4d054aa696aaa7ff76880a2f to finish bootstrapping
I20240501 14:08:57.753473 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 84c7d91c82524b69a9184fcf1949b5b0 to finish bootstrapping
I20240501 14:08:57.758174 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 73ea0b7a5e4c45a3814d46e74d62331a to finish bootstrapping
I20240501 14:08:57.763307 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 0c0d3d3aae054876a0a522a8325ddd02 to finish bootstrapping
I20240501 14:08:57.768900 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 6b961a167ae64dbc9d102ce5ddf721f3 to finish bootstrapping
W20240501 14:08:57.773159 9063 tablet.cc:2369] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:08:57.773959 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 9a65600939334f6db8dcca7f07923ece to finish bootstrapping
I20240501 14:08:57.779187 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 9458
W20240501 14:08:57.836941 9456 tablet.cc:2369] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:08:58.105188 9621 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 14:08:58.161216 9081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.153.199:42105: connect: Connection refused (error 111)
W20240501 14:08:58.162098 9081 consensus_peers.cc:485] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Status: Network error: Client connection negotiation failed: client connection to 127.6.153.199:42105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 14:08:58.174997 9621 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:08:58.180442 9640 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 14:09:00.453110 9081 consensus_peers.cc:485] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Status: Network error: Client connection negotiation failed: client connection to 127.6.153.199:42105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240501 14:09:01.538970 8612 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40168:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:01.546386 9127 tablet_service.cc:1467] Processing CreateTablet for tablet 4323f538490b4bf99fa7f96f80c8a88f (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.546722 9127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4323f538490b4bf99fa7f96f80c8a88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.546746 9126 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9e94ea8d834320a06db3087864a08c (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.547005 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9e94ea8d834320a06db3087864a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.547139 9258 tablet_service.cc:1467] Processing CreateTablet for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.547434 9258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b39b2a74a9c4dd78fc5541b3ce52772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.547643 9248 tablet_service.cc:1467] Processing CreateTablet for tablet 4323f538490b4bf99fa7f96f80c8a88f (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.547864 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4323f538490b4bf99fa7f96f80c8a88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.549234 9660 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:01.550357 9661 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:01.550441 8996 tablet_service.cc:1467] Processing CreateTablet for tablet 4323f538490b4bf99fa7f96f80c8a88f (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.550729 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4323f538490b4bf99fa7f96f80c8a88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.550698 8994 tablet_service.cc:1467] Processing CreateTablet for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.550908 8994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b39b2a74a9c4dd78fc5541b3ce52772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.550451 8995 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9e94ea8d834320a06db3087864a08c (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.551671 8995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9e94ea8d834320a06db3087864a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.551939 9388 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9e94ea8d834320a06db3087864a08c (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.552057 9661 tablet_bootstrap.cc:654] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.552201 9388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9e94ea8d834320a06db3087864a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.553174 9661 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:01.553259 9661 ts_tablet_manager.cc:1367] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:01.553808 9661 raft_consensus.cc:348] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.553930 9661 raft_consensus.cc:374] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.553958 9661 raft_consensus.cc:729] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.554088 9661 consensus_queue.cc:260] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.554385 9661 ts_tablet_manager.cc:1398] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:01.554549 9661 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:01.554649 9383 tablet_service.cc:1467] Processing CreateTablet for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.554898 9383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b39b2a74a9c4dd78fc5541b3ce52772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.555516 8734 tablet_service.cc:1467] Processing CreateTablet for tablet 4323f538490b4bf99fa7f96f80c8a88f (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.556315 9661 tablet_bootstrap.cc:654] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.556402 9660 tablet_bootstrap.cc:654] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.556514 8734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4323f538490b4bf99fa7f96f80c8a88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.557112 9663 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:01.557508 9665 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:01.558085 9663 tablet_bootstrap.cc:654] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.558362 9665 tablet_bootstrap.cc:654] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.559015 9661 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:01.559108 9661 ts_tablet_manager.cc:1367] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240501 14:09:01.559268 8733 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9e94ea8d834320a06db3087864a08c (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.559543 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9e94ea8d834320a06db3087864a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.559568 9661 raft_consensus.cc:348] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.559665 9661 raft_consensus.cc:374] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.559693 9661 raft_consensus.cc:729] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.559706 9660 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:01.559794 9660 ts_tablet_manager.cc:1367] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240501 14:09:01.559815 9661 consensus_queue.cc:260] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.560189 9661 ts_tablet_manager.cc:1398] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:01.560242 9663 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:01.560317 9663 ts_tablet_manager.cc:1367] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:01.560382 9660 raft_consensus.cc:348] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.560599 9660 raft_consensus.cc:374] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.560668 9660 raft_consensus.cc:729] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.560835 9660 consensus_queue.cc:260] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.560868 9663 raft_consensus.cc:348] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.560982 9663 raft_consensus.cc:374] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.561050 9663 raft_consensus.cc:729] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.561146 9660 ts_tablet_manager.cc:1398] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:01.561229 9663 consensus_queue.cc:260] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.561666 9663 ts_tablet_manager.cc:1398] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:01.561724 9660 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:01.562431 9665 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:01.562539 9665 ts_tablet_manager.cc:1367] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 14:09:01.562783 9660 tablet_bootstrap.cc:654] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.563072 9665 raft_consensus.cc:348] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.563210 9665 raft_consensus.cc:374] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.563257 9665 raft_consensus.cc:729] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.563441 9663 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:01.563474 9665 consensus_queue.cc:260] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.563594 8732 tablet_service.cc:1467] Processing CreateTablet for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.563850 8732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b39b2a74a9c4dd78fc5541b3ce52772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.563871 9665 ts_tablet_manager.cc:1398] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:01.564018 9665 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:01.564621 9663 tablet_bootstrap.cc:654] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.565063 9665 tablet_bootstrap.cc:654] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.565719 9660 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:01.565794 9660 ts_tablet_manager.cc:1367] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 14:09:01.565948 9665 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:01.566013 9665 ts_tablet_manager.cc:1367] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:01.566190 9660 raft_consensus.cc:348] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.566289 9660 raft_consensus.cc:374] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.566320 9660 raft_consensus.cc:729] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.566406 9665 raft_consensus.cc:348] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.566428 9660 consensus_queue.cc:260] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.566499 9665 raft_consensus.cc:374] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.566530 9665 raft_consensus.cc:729] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.566668 9660 ts_tablet_manager.cc:1398] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:01.566629 9665 consensus_queue.cc:260] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.566946 9665 ts_tablet_manager.cc:1398] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:01.570294 9663 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:01.570382 9663 ts_tablet_manager.cc:1367] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240501 14:09:01.570816 9677 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:01.570834 9663 raft_consensus.cc:348] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.570932 9663 raft_consensus.cc:374] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.570959 9663 raft_consensus.cc:729] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.571084 9663 consensus_queue.cc:260] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.571355 9663 ts_tablet_manager.cc:1398] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:01.571511 9663 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:01.572516 9663 tablet_bootstrap.cc:654] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.574028 8865 tablet_service.cc:1467] Processing CreateTablet for tablet 4323f538490b4bf99fa7f96f80c8a88f (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.573923 8864 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9e94ea8d834320a06db3087864a08c (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.574399 9677 tablet_bootstrap.cc:654] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.573923 8863 tablet_service.cc:1467] Processing CreateTablet for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 (DEFAULT_TABLE table=rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:01.575042 8864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9e94ea8d834320a06db3087864a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.575203 9677 log.cc:824] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:01.575423 8865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4323f538490b4bf99fa7f96f80c8a88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.575872 9663 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:01.576030 9663 ts_tablet_manager.cc:1367] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240501 14:09:01.576537 9663 raft_consensus.cc:348] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.576717 9663 raft_consensus.cc:374] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.576802 8863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b39b2a74a9c4dd78fc5541b3ce52772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:01.576830 9663 raft_consensus.cc:729] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.576962 9663 consensus_queue.cc:260] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.577373 9663 ts_tablet_manager.cc:1398] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:01.577450 9677 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:01.577653 9677 ts_tablet_manager.cc:1367] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240501 14:09:01.581003 9677 raft_consensus.cc:348] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.581230 9677 raft_consensus.cc:374] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.581297 9677 raft_consensus.cc:729] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.581808 9677 consensus_queue.cc:260] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.582711 9680 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:01.582784 9677 ts_tablet_manager.cc:1398] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 14:09:01.583137 9677 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:01.584220 9677 tablet_bootstrap.cc:654] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.585116 9680 tablet_bootstrap.cc:654] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.585465 9677 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:01.585553 9677 ts_tablet_manager.cc:1367] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:01.585938 9680 log.cc:824] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:01.586102 9677 raft_consensus.cc:348] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.586223 9677 raft_consensus.cc:374] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.586262 9677 raft_consensus.cc:729] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.586422 9677 consensus_queue.cc:260] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.586668 9677 ts_tablet_manager.cc:1398] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:01.586843 9677 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:01.587688 9677 tablet_bootstrap.cc:654] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.587747 9680 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:01.587930 9680 ts_tablet_manager.cc:1367] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240501 14:09:01.591470 9680 raft_consensus.cc:348] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.591720 9680 raft_consensus.cc:374] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.591784 9680 raft_consensus.cc:729] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.592276 9680 consensus_queue.cc:260] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.593155 9680 ts_tablet_manager.cc:1398] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20240501 14:09:01.593294 8932 tablet.cc:2369] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:09:01.593421 9680 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:01.594360 9680 tablet_bootstrap.cc:654] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.595620 9677 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:01.595660 9680 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:01.595633 9671 raft_consensus.cc:482] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:01.595721 9680 ts_tablet_manager.cc:1367] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:01.595719 9677 ts_tablet_manager.cc:1367] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240501 14:09:01.595738 9671 raft_consensus.cc:504] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.596155 9677 raft_consensus.cc:348] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.596273 9677 raft_consensus.cc:374] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.596225 9680 raft_consensus.cc:348] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.596303 9677 raft_consensus.cc:729] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.596311 9680 raft_consensus.cc:374] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.596341 9680 raft_consensus.cc:729] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.596427 9677 consensus_queue.cc:260] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.596477 9680 consensus_queue.cc:260] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.596721 9677 ts_tablet_manager.cc:1398] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:01.596807 9680 ts_tablet_manager.cc:1398] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:01.596946 9680 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:01.597111 9671 leader_election.cc:288] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:01.598006 9680 tablet_bootstrap.cc:654] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:01.600340 9680 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:01.600415 9680 ts_tablet_manager.cc:1367] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:01.601245 9680 raft_consensus.cc:348] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.601359 9680 raft_consensus.cc:374] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:01.601389 9680 raft_consensus.cc:729] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:01.601507 9680 consensus_queue.cc:260] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.601789 9680 ts_tablet_manager.cc:1398] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:01.603475 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:01.603739 8885 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:01.605357 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:01.605628 8754 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:01.605969 8950 leader_election.cc:302] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 25737230978048598f426b1a99b27441, 84c7d91c82524b69a9184fcf1949b5b0, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:01.606017 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:01.606151 9409 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:01.606282 9671 raft_consensus.cc:2781] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:01.606343 9671 raft_consensus.cc:482] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:01.606379 9671 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.606846 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:01.606994 9147 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:01.607355 9671 raft_consensus.cc:504] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.607800 9671 leader_election.cc:288] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 election: Requested vote from peers 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:01.608072 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:01.608116 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:01.608115 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:01.608117 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:01.608179 8754 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.608211 9147 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.608215 9409 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.608215 8885 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.609135 8885 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:01.609135 9147 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:01.609140 9409 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:01.609140 8754 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:01.609444 8949 leader_election.cc:302] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6b961a167ae64dbc9d102ce5ddf721f3, 73ea0b7a5e4c45a3814d46e74d62331a, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:01.609620 9671 raft_consensus.cc:2781] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:01.609830 9671 raft_consensus.cc:686] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Becoming Leader. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Running, Role: LEADER
I20240501 14:09:01.610283 9671 consensus_queue.cc:237] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:01.612802 8610 catalog_manager.cc:5484] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:01.638228 9671 raft_consensus.cc:482] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:01.638375 9671 raft_consensus.cc:504] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.639431 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4323f538490b4bf99fa7f96f80c8a88f" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:01.639480 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4323f538490b4bf99fa7f96f80c8a88f" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:01.639616 8754 raft_consensus.cc:2454] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:01.639627 8885 raft_consensus.cc:2454] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:01.639690 9671 leader_election.cc:288] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983)
I20240501 14:09:01.640048 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4323f538490b4bf99fa7f96f80c8a88f" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:01.640100 8949 leader_election.cc:302] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 25737230978048598f426b1a99b27441, 84c7d91c82524b69a9184fcf1949b5b0, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:01.640200 9147 raft_consensus.cc:2454] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:01.640262 9671 raft_consensus.cc:2781] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:01.640312 9671 raft_consensus.cc:482] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:01.640354 9671 raft_consensus.cc:3037] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.641207 9671 raft_consensus.cc:504] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.641636 9671 leader_election.cc:288] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 election: Requested vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983)
I20240501 14:09:01.641882 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4323f538490b4bf99fa7f96f80c8a88f" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:01.641945 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4323f538490b4bf99fa7f96f80c8a88f" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:01.642004 8754 raft_consensus.cc:3037] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.642058 8885 raft_consensus.cc:3037] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.642057 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4323f538490b4bf99fa7f96f80c8a88f" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:01.642163 9147 raft_consensus.cc:3037] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.643011 8754 raft_consensus.cc:2454] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:01.643056 9147 raft_consensus.cc:2454] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:01.643187 8885 raft_consensus.cc:2454] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:01.643348 8951 leader_election.cc:302] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 25737230978048598f426b1a99b27441, 73ea0b7a5e4c45a3814d46e74d62331a, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:01.643520 9671 raft_consensus.cc:2781] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:01.643605 9671 raft_consensus.cc:686] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Becoming Leader. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Running, Role: LEADER
I20240501 14:09:01.643749 9671 consensus_queue.cc:237] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } }
I20240501 14:09:01.644241 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4323f538490b4bf99fa7f96f80c8a88f" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:09:01.644325 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4323f538490b4bf99fa7f96f80c8a88f" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:01.644379 9278 raft_consensus.cc:2454] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:01.644449 9277 raft_consensus.cc:3037] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.645398 9277 raft_consensus.cc:2454] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:01.645742 8610 catalog_manager.cc:5484] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } }
W20240501 14:09:01.663372 8801 tablet.cc:2369] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:09:01.775058 9669 raft_consensus.cc:482] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:01.775211 9669 raft_consensus.cc:504] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.776603 9669 leader_election.cc:288] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:01.782214 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:09:01.782214 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:01.782384 8885 raft_consensus.cc:2454] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:01.782385 9015 raft_consensus.cc:2454] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:01.782563 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:01.782716 8754 raft_consensus.cc:2454] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:01.782886 9211 leader_election.cc:302] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 84c7d91c82524b69a9184fcf1949b5b0, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:01.783197 9669 raft_consensus.cc:2781] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:01.783250 9669 raft_consensus.cc:482] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:01.783275 9669 raft_consensus.cc:3037] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.784287 9669 raft_consensus.cc:504] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.784543 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:01.784665 9409 raft_consensus.cc:2454] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:01.784765 9669 leader_election.cc:288] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 election: Requested vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:01.785058 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:01.785041 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:01.785164 9015 raft_consensus.cc:3037] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.785164 8754 raft_consensus.cc:3037] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.785562 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:01.785542 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:01.785665 9409 raft_consensus.cc:3037] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.785665 8885 raft_consensus.cc:3037] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:01.786078 9015 raft_consensus.cc:2454] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:01.786078 8754 raft_consensus.cc:2454] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:01.786525 8885 raft_consensus.cc:2454] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:01.786532 9409 raft_consensus.cc:2454] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:01.786899 9212 leader_election.cc:302] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 25737230978048598f426b1a99b27441, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:01.787096 9669 raft_consensus.cc:2781] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:01.787200 9669 raft_consensus.cc:686] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Becoming Leader. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Running, Role: LEADER
I20240501 14:09:01.787514 9669 consensus_queue.cc:237] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:01.789748 8610 catalog_manager.cc:5484] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197). New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:01.900152 9704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:01.900396 9704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:01.903548 9704 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 14:09:01.991469 9277 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:01.991973 9409 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:01.992063 9671 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:01.992359 8754 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:01.992638 8885 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:01.992969 9669 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:01.996721 8752 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:01.996834 9146 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:01.997308 9694 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:01.998581 9664 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:01.999639 9015 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:01.999680 9409 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.000461 9694 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.001889 9147 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.001876 9664 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.003245 8885 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.003420 9671 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.003719 9671 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.003777 8754 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.004318 8883 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.004525 9694 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.004640 9671 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.009936 9669 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.011718 9732 mvcc.cc:205] Tried to move back new op lower bound from 7022889132013096960 to 7022889130438864896. Current Snapshot: MvccSnapshot[applied={T|T < 7022889132013096960}]
I20240501 14:09:02.016096 9671 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.026674 9746 mvcc.cc:205] Tried to move back new op lower bound from 7022889131990622208 to 7022889131164368896. Current Snapshot: MvccSnapshot[applied={T|T < 7022889131990622208}]
I20240501 14:09:02.026860 9731 mvcc.cc:205] Tried to move back new op lower bound from 7022889132012519424 to 7022889130574925824. Current Snapshot: MvccSnapshot[applied={T|T < 7022889131991461888}]
I20240501 14:09:02.037664 9757 mvcc.cc:205] Tried to move back new op lower bound from 7022889131990622208 to 7022889131164368896. Current Snapshot: MvccSnapshot[applied={T|T < 7022889131990622208}]
I20240501 14:09:02.053537 9745 mvcc.cc:205] Tried to move back new op lower bound from 7022889132035174400 to 7022889130438864896. Current Snapshot: MvccSnapshot[applied={T|T < 7022889132013096960}]
I20240501 14:09:02.058570 9743 mvcc.cc:205] Tried to move back new op lower bound from 7022889132012519424 to 7022889130574925824. Current Snapshot: MvccSnapshot[applied={T|T < 7022889131991461888}]
INSERT report
rows total: 2000
time total: 158.481 ms
time per row: 0.0792405 ms
I20240501 14:09:02.107453 8611 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40168:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:02.112709 9126 tablet_service.cc:1467] Processing CreateTablet for tablet 560c845cf74c4df1bf33732abc84620d (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.112766 9248 tablet_service.cc:1467] Processing CreateTablet for tablet 560c845cf74c4df1bf33732abc84620d (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.113047 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560c845cf74c4df1bf33732abc84620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.113080 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560c845cf74c4df1bf33732abc84620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.114920 9783 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:02.115027 9784 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:02.115778 8996 tablet_service.cc:1467] Processing CreateTablet for tablet a6b7f72cb79d480a8a39e20c51d922f9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.115846 9783 tablet_bootstrap.cc:654] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.115967 9784 tablet_bootstrap.cc:654] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.116056 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b7f72cb79d480a8a39e20c51d922f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.116806 9126 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2fe0f030204ecca740b14dfd6605ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.117033 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2fe0f030204ecca740b14dfd6605ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.117815 9787 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:02.117997 8734 tablet_service.cc:1467] Processing CreateTablet for tablet 560c845cf74c4df1bf33732abc84620d (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.118245 8734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560c845cf74c4df1bf33732abc84620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.118678 9787 tablet_bootstrap.cc:654] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.119590 8733 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2fe0f030204ecca740b14dfd6605ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.119827 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2fe0f030204ecca740b14dfd6605ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.119900 9784 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:02.119910 9787 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:02.119985 9784 ts_tablet_manager.cc:1367] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 14:09:02.119998 9787 ts_tablet_manager.cc:1367] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:02.120335 9787 raft_consensus.cc:348] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.120436 9787 raft_consensus.cc:374] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.120466 9787 raft_consensus.cc:729] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.120412 9784 raft_consensus.cc:348] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.120527 9784 raft_consensus.cc:374] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.120555 9784 raft_consensus.cc:729] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.120576 9787 consensus_queue.cc:260] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.120689 9784 consensus_queue.cc:260] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.120980 9784 ts_tablet_manager.cc:1398] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:02.121146 9784 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:02.121984 9784 tablet_bootstrap.cc:654] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.121965 8732 tablet_service.cc:1467] Processing CreateTablet for tablet a6b7f72cb79d480a8a39e20c51d922f9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.122195 8732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b7f72cb79d480a8a39e20c51d922f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.122543 9789 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:02.122707 9787 ts_tablet_manager.cc:1398] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:09:02.123425 9789 tablet_bootstrap.cc:654] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.123571 9127 tablet_service.cc:1467] Processing CreateTablet for tablet a6b7f72cb79d480a8a39e20c51d922f9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.123790 9127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b7f72cb79d480a8a39e20c51d922f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.125016 8864 tablet_service.cc:1467] Processing CreateTablet for tablet 560c845cf74c4df1bf33732abc84620d (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.125269 8864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560c845cf74c4df1bf33732abc84620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.125458 9383 tablet_service.cc:1467] Processing CreateTablet for tablet 560c845cf74c4df1bf33732abc84620d (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.125499 9258 tablet_service.cc:1467] Processing CreateTablet for tablet a6b7f72cb79d480a8a39e20c51d922f9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.125732 9258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b7f72cb79d480a8a39e20c51d922f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.125732 9383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560c845cf74c4df1bf33732abc84620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.125825 8865 tablet_service.cc:1467] Processing CreateTablet for tablet a6b7f72cb79d480a8a39e20c51d922f9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.126037 8865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b7f72cb79d480a8a39e20c51d922f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.127532 8863 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2fe0f030204ecca740b14dfd6605ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.127743 8863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2fe0f030204ecca740b14dfd6605ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.127933 9388 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2fe0f030204ecca740b14dfd6605ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.128144 9388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2fe0f030204ecca740b14dfd6605ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.128759 9248 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2fe0f030204ecca740b14dfd6605ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.128976 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2fe0f030204ecca740b14dfd6605ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.129743 9784 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:02.129822 9784 ts_tablet_manager.cc:1367] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:09:02.130276 9784 raft_consensus.cc:348] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.130383 9784 raft_consensus.cc:374] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.130414 9784 raft_consensus.cc:729] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.130560 9784 consensus_queue.cc:260] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.130826 9784 ts_tablet_manager.cc:1398] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.130995 9784 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:02.131374 9792 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:02.131680 9789 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:02.131750 9789 ts_tablet_manager.cc:1367] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240501 14:09:02.131883 9784 tablet_bootstrap.cc:654] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.132086 9789 raft_consensus.cc:348] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.132164 9789 raft_consensus.cc:374] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.132184 9789 raft_consensus.cc:729] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.132215 9792 tablet_bootstrap.cc:654] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.132290 9789 consensus_queue.cc:260] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.132648 9789 ts_tablet_manager.cc:1398] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.132767 9783 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:02.132792 9789 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:02.132848 9783 ts_tablet_manager.cc:1367] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240501 14:09:02.133180 9792 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:02.133253 9792 ts_tablet_manager.cc:1367] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:02.133275 9783 raft_consensus.cc:348] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.133384 9783 raft_consensus.cc:374] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.133414 9783 raft_consensus.cc:729] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.133529 9789 tablet_bootstrap.cc:654] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.133541 9783 consensus_queue.cc:260] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.133656 9792 raft_consensus.cc:348] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.133749 9792 raft_consensus.cc:374] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.133777 9792 raft_consensus.cc:729] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.133805 9783 ts_tablet_manager.cc:1398] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.133901 9792 consensus_queue.cc:260] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.133975 9783 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:02.134174 9792 ts_tablet_manager.cc:1398] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.134342 9792 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:02.134971 9783 tablet_bootstrap.cc:654] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.135124 9792 tablet_bootstrap.cc:654] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.136456 9792 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:02.136538 9792 ts_tablet_manager.cc:1367] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 14:09:02.136948 9792 raft_consensus.cc:348] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.137060 9792 raft_consensus.cc:374] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.137096 9792 raft_consensus.cc:729] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.137212 9792 consensus_queue.cc:260] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.137466 9792 ts_tablet_manager.cc:1398] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:02.137550 9784 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:02.137615 9792 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:02.137626 9784 ts_tablet_manager.cc:1367] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240501 14:09:02.138026 9784 raft_consensus.cc:348] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.138139 9784 raft_consensus.cc:374] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.138166 9784 raft_consensus.cc:729] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.138410 9792 tablet_bootstrap.cc:654] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.138434 9784 consensus_queue.cc:260] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.138820 9784 ts_tablet_manager.cc:1398] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:02.139180 9793 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:02.139446 9783 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:02.139535 9783 ts_tablet_manager.cc:1367] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:09:02.139966 9783 raft_consensus.cc:348] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.140089 9783 raft_consensus.cc:374] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.140125 9783 raft_consensus.cc:729] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.140249 9783 consensus_queue.cc:260] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.140506 9783 ts_tablet_manager.cc:1398] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.140656 9793 tablet_bootstrap.cc:654] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.140893 9783 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:02.141784 9783 tablet_bootstrap.cc:654] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.142036 9793 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:02.142132 9793 ts_tablet_manager.cc:1367] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:02.142598 9793 raft_consensus.cc:348] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.142715 9793 raft_consensus.cc:374] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.142747 9793 raft_consensus.cc:729] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.142858 9783 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:02.142899 9793 consensus_queue.cc:260] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.142958 9783 ts_tablet_manager.cc:1367] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:02.143141 9792 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:02.143218 9792 ts_tablet_manager.cc:1367] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240501 14:09:02.143499 9783 raft_consensus.cc:348] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.143602 9783 raft_consensus.cc:374] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.143630 9783 raft_consensus.cc:729] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.143669 9792 raft_consensus.cc:348] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.143777 9792 raft_consensus.cc:374] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.143752 9783 consensus_queue.cc:260] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.143810 9792 raft_consensus.cc:729] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.143918 9792 consensus_queue.cc:260] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.144026 9783 ts_tablet_manager.cc:1398] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.144196 9792 ts_tablet_manager.cc:1398] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.144541 9793 ts_tablet_manager.cc:1398] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:02.144723 9793 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:02.145203 9789 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:02.145308 9789 ts_tablet_manager.cc:1367] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20240501 14:09:02.145769 9789 raft_consensus.cc:348] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.145892 9789 raft_consensus.cc:374] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.145921 9789 raft_consensus.cc:729] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.146040 9789 consensus_queue.cc:260] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.146492 9789 ts_tablet_manager.cc:1398] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 14:09:02.146661 9789 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:02.147812 9789 tablet_bootstrap.cc:654] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.148104 9793 tablet_bootstrap.cc:654] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.148813 9789 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:02.148891 9789 ts_tablet_manager.cc:1367] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:02.149058 9793 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:02.149140 9793 ts_tablet_manager.cc:1367] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:09:02.149575 9793 raft_consensus.cc:348] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.149662 9793 raft_consensus.cc:374] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.149685 9793 raft_consensus.cc:729] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.149791 9793 consensus_queue.cc:260] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.150019 9793 ts_tablet_manager.cc:1398] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.158587 9789 raft_consensus.cc:348] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.158762 9789 raft_consensus.cc:374] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.158829 9789 raft_consensus.cc:729] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.158967 9789 consensus_queue.cc:260] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.159323 9789 ts_tablet_manager.cc:1398] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20240501 14:09:02.172427 9681 raft_consensus.cc:482] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:02.172601 9681 raft_consensus.cc:504] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.173962 9681 leader_election.cc:288] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:02.178678 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b7f72cb79d480a8a39e20c51d922f9" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:09:02.178856 9015 raft_consensus.cc:2454] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.179219 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b7f72cb79d480a8a39e20c51d922f9" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:09:02.179378 9277 raft_consensus.cc:2454] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.179646 8689 leader_election.cc:302] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 25737230978048598f426b1a99b27441, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:02.179839 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b7f72cb79d480a8a39e20c51d922f9" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:02.179900 9681 raft_consensus.cc:2781] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:02.179970 9681 raft_consensus.cc:482] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:02.179970 9147 raft_consensus.cc:2454] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.180011 9681 raft_consensus.cc:3037] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.180301 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b7f72cb79d480a8a39e20c51d922f9" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:02.180414 8885 raft_consensus.cc:2454] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.180982 9681 raft_consensus.cc:504] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.181406 9681 leader_election.cc:288] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 election: Requested vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:02.181665 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b7f72cb79d480a8a39e20c51d922f9" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:02.181695 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b7f72cb79d480a8a39e20c51d922f9" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:02.181782 9015 raft_consensus.cc:3037] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.181784 9277 raft_consensus.cc:3037] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.181751 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b7f72cb79d480a8a39e20c51d922f9" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:02.181828 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b7f72cb79d480a8a39e20c51d922f9" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:02.181864 9147 raft_consensus.cc:3037] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.181932 8885 raft_consensus.cc:3037] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.182687 9015 raft_consensus.cc:2454] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.182687 9277 raft_consensus.cc:2454] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.182780 9147 raft_consensus.cc:2454] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.182797 8885 raft_consensus.cc:2454] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.183027 8690 leader_election.cc:302] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 25737230978048598f426b1a99b27441, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:02.183190 9681 raft_consensus.cc:2781] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:02.183440 9681 raft_consensus.cc:686] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Becoming Leader. State: Replica: 25737230978048598f426b1a99b27441, State: Running, Role: LEADER
I20240501 14:09:02.183745 9681 consensus_queue.cc:237] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:02.186148 8611 catalog_manager.cc:5484] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25737230978048598f426b1a99b27441 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:02.296773 9673 raft_consensus.cc:482] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:02.296918 9673 raft_consensus.cc:504] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.297611 9673 leader_election.cc:288] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:02.298297 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2fe0f030204ecca740b14dfd6605ff" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:09:02.298476 9277 raft_consensus.cc:2454] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b961a167ae64dbc9d102ce5ddf721f3 in term 0.
I20240501 14:09:02.298471 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2fe0f030204ecca740b14dfd6605ff" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:02.298640 9147 raft_consensus.cc:2454] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b961a167ae64dbc9d102ce5ddf721f3 in term 0.
I20240501 14:09:02.299286 9344 leader_election.cc:302] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 6b961a167ae64dbc9d102ce5ddf721f3, 73ea0b7a5e4c45a3814d46e74d62331a; no voters:
I20240501 14:09:02.299494 9673 raft_consensus.cc:2781] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:02.299552 9673 raft_consensus.cc:482] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:02.299608 9673 raft_consensus.cc:3037] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.300649 9673 raft_consensus.cc:504] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.301132 9673 leader_election.cc:288] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [CANDIDATE]: Term 1 election: Requested vote from peers 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:02.301379 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2fe0f030204ecca740b14dfd6605ff" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:02.301450 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2fe0f030204ecca740b14dfd6605ff" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:02.301528 9147 raft_consensus.cc:3037] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.301584 9277 raft_consensus.cc:3037] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.302433 9277 raft_consensus.cc:2454] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b961a167ae64dbc9d102ce5ddf721f3 in term 1.
I20240501 14:09:02.302547 9147 raft_consensus.cc:2454] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b961a167ae64dbc9d102ce5ddf721f3 in term 1.
I20240501 14:09:02.302801 9344 leader_election.cc:302] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 6b961a167ae64dbc9d102ce5ddf721f3, 73ea0b7a5e4c45a3814d46e74d62331a; no voters:
I20240501 14:09:02.303038 9673 raft_consensus.cc:2781] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:02.303004 8752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2fe0f030204ecca740b14dfd6605ff" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:02.303093 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2fe0f030204ecca740b14dfd6605ff" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:02.303143 8752 raft_consensus.cc:2454] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b961a167ae64dbc9d102ce5ddf721f3 in term 0.
I20240501 14:09:02.303212 8754 raft_consensus.cc:3037] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.303231 9673 raft_consensus.cc:686] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 LEADER]: Becoming Leader. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Running, Role: LEADER
I20240501 14:09:02.303591 9673 consensus_queue.cc:237] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.303670 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2fe0f030204ecca740b14dfd6605ff" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:02.303799 8885 raft_consensus.cc:2454] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b961a167ae64dbc9d102ce5ddf721f3 in term 0.
I20240501 14:09:02.303881 8883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2fe0f030204ecca740b14dfd6605ff" candidate_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:02.303958 8883 raft_consensus.cc:3037] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.304164 8754 raft_consensus.cc:2454] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b961a167ae64dbc9d102ce5ddf721f3 in term 1.
I20240501 14:09:02.304754 8883 raft_consensus.cc:2454] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b961a167ae64dbc9d102ce5ddf721f3 in term 1.
I20240501 14:09:02.306661 8611 catalog_manager.cc:5484] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198). New cstate: current_term: 1 leader_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:02.334748 9681 raft_consensus.cc:482] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:02.334889 9681 raft_consensus.cc:504] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.335638 9681 leader_election.cc:288] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:02.336428 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560c845cf74c4df1bf33732abc84620d" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:02.336742 9147 raft_consensus.cc:2454] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.337373 8883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560c845cf74c4df1bf33732abc84620d" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:02.337523 8883 raft_consensus.cc:2454] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.337450 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560c845cf74c4df1bf33732abc84620d" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:09:02.337806 9277 raft_consensus.cc:2454] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.337865 8687 leader_election.cc:302] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 25737230978048598f426b1a99b27441, 73ea0b7a5e4c45a3814d46e74d62331a, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:02.338066 9681 raft_consensus.cc:2781] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:02.338128 9681 raft_consensus.cc:482] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:02.338176 9681 raft_consensus.cc:3037] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.339622 9681 raft_consensus.cc:504] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.340480 8883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560c845cf74c4df1bf33732abc84620d" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:02.340603 8883 raft_consensus.cc:3037] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.340762 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560c845cf74c4df1bf33732abc84620d" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:02.340889 9277 raft_consensus.cc:3037] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.341554 8883 raft_consensus.cc:2454] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.341553 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560c845cf74c4df1bf33732abc84620d" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:02.341670 9147 raft_consensus.cc:3037] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.341753 9277 raft_consensus.cc:2454] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.341995 9681 leader_election.cc:288] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 election: Requested vote from peers 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:02.342151 8689 leader_election.cc:302] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 25737230978048598f426b1a99b27441, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:02.342314 9681 raft_consensus.cc:2781] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:02.342465 9681 raft_consensus.cc:686] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Becoming Leader. State: Replica: 25737230978048598f426b1a99b27441, State: Running, Role: LEADER
I20240501 14:09:02.342579 9147 raft_consensus.cc:2454] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.342659 9681 consensus_queue.cc:237] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.343813 9408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560c845cf74c4df1bf33732abc84620d" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:02.343832 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "560c845cf74c4df1bf33732abc84620d" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:02.343964 9408 raft_consensus.cc:2454] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.345013 8611 catalog_manager.cc:5484] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25737230978048598f426b1a99b27441 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:02.461459 9818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:02.461727 9818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:02.464828 9818 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 14:09:02.549753 9277 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.549975 8883 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.550431 9681 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.550935 9809 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.551900 8885 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.552299 9409 raft_consensus.cc:3037] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.553607 9147 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.553963 9278 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.554410 8754 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.554527 8885 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.554980 9814 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.555428 9814 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.555691 9814 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.555923 9814 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.556741 9809 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.557543 9147 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.558001 9276 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:09:02.563627 9681 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.564005 9681 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.565460 9015 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.565871 9681 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.570338 9145 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.572402 9809 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:02.600288 9409 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:02.603914 9681 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 155.861 ms
time per row: 0.0779306 ms
I20240501 14:09:02.664328 8603 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40168:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:02.670387 8734 tablet_service.cc:1467] Processing CreateTablet for tablet d21fe5dd65fe4fbea85fecc8ce19bc43 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.670486 8996 tablet_service.cc:1467] Processing CreateTablet for tablet d21fe5dd65fe4fbea85fecc8ce19bc43 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.670746 8734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21fe5dd65fe4fbea85fecc8ce19bc43. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.670792 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21fe5dd65fe4fbea85fecc8ce19bc43. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.672799 9879 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:02.672881 9880 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:02.673321 9126 tablet_service.cc:1467] Processing CreateTablet for tablet e72446fcffed439c8653937330e5b8ee (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.673506 9127 tablet_service.cc:1467] Processing CreateTablet for tablet d21fe5dd65fe4fbea85fecc8ce19bc43 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.673607 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72446fcffed439c8653937330e5b8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.673748 9127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21fe5dd65fe4fbea85fecc8ce19bc43. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.673800 9880 tablet_bootstrap.cc:654] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.674096 9879 tablet_bootstrap.cc:654] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.675117 9879 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:02.675269 9879 ts_tablet_manager.cc:1367] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:02.675551 9883 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:02.675806 9879 raft_consensus.cc:348] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.675841 9248 tablet_service.cc:1467] Processing CreateTablet for tablet 188101b7f3784e48975a673952b2bc63 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.675935 9879 raft_consensus.cc:374] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.676115 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188101b7f3784e48975a673952b2bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.675973 9879 raft_consensus.cc:729] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.676604 9880 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:02.676679 9880 ts_tablet_manager.cc:1367] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:02.676623 9879 consensus_queue.cc:260] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.676913 9879 ts_tablet_manager.cc:1398] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.677142 9880 raft_consensus.cc:348] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.677246 9880 raft_consensus.cc:374] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.677274 9880 raft_consensus.cc:729] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.677397 9880 consensus_queue.cc:260] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.677656 9880 ts_tablet_manager.cc:1398] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:02.678537 9883 tablet_bootstrap.cc:654] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.678861 8863 tablet_service.cc:1467] Processing CreateTablet for tablet d21fe5dd65fe4fbea85fecc8ce19bc43 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.679142 8863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21fe5dd65fe4fbea85fecc8ce19bc43. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.681816 9886 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:02.681931 8865 tablet_service.cc:1467] Processing CreateTablet for tablet 188101b7f3784e48975a673952b2bc63 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.682277 8865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188101b7f3784e48975a673952b2bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.682725 8734 tablet_service.cc:1467] Processing CreateTablet for tablet 188101b7f3784e48975a673952b2bc63 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.683010 8734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188101b7f3784e48975a673952b2bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.683113 9886 tablet_bootstrap.cc:654] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.682725 8732 tablet_service.cc:1467] Processing CreateTablet for tablet e72446fcffed439c8653937330e5b8ee (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.683598 8732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72446fcffed439c8653937330e5b8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.684193 9886 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:02.684285 9886 ts_tablet_manager.cc:1367] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:02.684697 9886 raft_consensus.cc:348] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.684793 9886 raft_consensus.cc:374] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.684823 9886 raft_consensus.cc:729] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.684940 9886 consensus_queue.cc:260] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.685181 9886 ts_tablet_manager.cc:1398] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.685343 9886 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:02.685422 9880 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:02.686215 9880 tablet_bootstrap.cc:654] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.686223 9886 tablet_bootstrap.cc:654] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.686450 8994 tablet_service.cc:1467] Processing CreateTablet for tablet e72446fcffed439c8653937330e5b8ee (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.686723 8994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72446fcffed439c8653937330e5b8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.683317 9383 tablet_service.cc:1467] Processing CreateTablet for tablet 188101b7f3784e48975a673952b2bc63 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.684417 9388 tablet_service.cc:1467] Processing CreateTablet for tablet d21fe5dd65fe4fbea85fecc8ce19bc43 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.687698 9383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188101b7f3784e48975a673952b2bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.687747 9886 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:02.687826 9886 ts_tablet_manager.cc:1367] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:09:02.687898 9388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21fe5dd65fe4fbea85fecc8ce19bc43. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.675841 9258 tablet_service.cc:1467] Processing CreateTablet for tablet e72446fcffed439c8653937330e5b8ee (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.688220 9258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72446fcffed439c8653937330e5b8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.688275 9886 raft_consensus.cc:348] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.688374 9886 raft_consensus.cc:374] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.688400 9886 raft_consensus.cc:729] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.688513 9886 consensus_queue.cc:260] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.689059 9879 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:02.689556 9886 ts_tablet_manager.cc:1398] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:09:02.689935 9879 tablet_bootstrap.cc:654] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.690054 9890 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:02.685153 9389 tablet_service.cc:1467] Processing CreateTablet for tablet e72446fcffed439c8653937330e5b8ee (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.690754 9389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72446fcffed439c8653937330e5b8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.691008 8996 tablet_service.cc:1467] Processing CreateTablet for tablet 188101b7f3784e48975a673952b2bc63 (DEFAULT_TABLE table=rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:02.691227 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188101b7f3784e48975a673952b2bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:02.692332 9890 tablet_bootstrap.cc:654] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.693284 9884 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:02.693987 9884 tablet_bootstrap.cc:654] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.694766 9879 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:02.694864 9879 ts_tablet_manager.cc:1367] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:09:02.694981 9884 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:02.695053 9884 ts_tablet_manager.cc:1367] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:02.695269 9880 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:02.695394 9880 ts_tablet_manager.cc:1367] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 14:09:02.695462 9879 raft_consensus.cc:348] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.695590 9879 raft_consensus.cc:374] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.695622 9879 raft_consensus.cc:729] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.695767 9879 consensus_queue.cc:260] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.695827 9880 raft_consensus.cc:348] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.695935 9880 raft_consensus.cc:374] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.695960 9880 raft_consensus.cc:729] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.696069 9880 consensus_queue.cc:260] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.696341 9880 ts_tablet_manager.cc:1398] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.696318 9884 raft_consensus.cc:348] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.696427 9884 raft_consensus.cc:374] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.696460 9884 raft_consensus.cc:729] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.696509 9880 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:02.696597 9884 consensus_queue.cc:260] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.696851 9884 ts_tablet_manager.cc:1398] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:02.697584 9880 tablet_bootstrap.cc:654] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.697031 9884 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:02.698921 9879 ts_tablet_manager.cc:1398] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:02.699681 9880 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:02.699762 9880 ts_tablet_manager.cc:1367] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:02.700172 9883 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:02.700184 9884 tablet_bootstrap.cc:654] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.700189 9880 raft_consensus.cc:348] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.700305 9880 raft_consensus.cc:374] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.700336 9880 raft_consensus.cc:729] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.700448 9883 ts_tablet_manager.cc:1367] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240501 14:09:02.700469 9880 consensus_queue.cc:260] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.700742 9880 ts_tablet_manager.cc:1398] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.701126 9883 raft_consensus.cc:348] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.701244 9883 raft_consensus.cc:374] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.701239 9884 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:02.701274 9883 raft_consensus.cc:729] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.701325 9884 ts_tablet_manager.cc:1367] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:02.701431 9883 consensus_queue.cc:260] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.701781 9883 ts_tablet_manager.cc:1398] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.701756 9884 raft_consensus.cc:348] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.701865 9884 raft_consensus.cc:374] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.701896 9884 raft_consensus.cc:729] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.701946 9883 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:02.702023 9884 consensus_queue.cc:260] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.702318 9884 ts_tablet_manager.cc:1398] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.702838 9883 tablet_bootstrap.cc:654] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.703554 9879 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:02.704161 9883 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:02.704238 9883 ts_tablet_manager.cc:1367] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:02.704648 9879 tablet_bootstrap.cc:654] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.704677 9883 raft_consensus.cc:348] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.704792 9883 raft_consensus.cc:374] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.704823 9883 raft_consensus.cc:729] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.704950 9883 consensus_queue.cc:260] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.705271 9883 ts_tablet_manager.cc:1398] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.705454 9890 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:02.705586 9890 ts_tablet_manager.cc:1367] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240501 14:09:02.705703 9879 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:02.705797 9879 ts_tablet_manager.cc:1367] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:02.706005 9890 raft_consensus.cc:348] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.706094 9890 raft_consensus.cc:374] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.706113 9890 raft_consensus.cc:729] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.706252 9890 consensus_queue.cc:260] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.706238 9879 raft_consensus.cc:348] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.706346 9879 raft_consensus.cc:374] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.706377 9879 raft_consensus.cc:729] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.706527 9890 ts_tablet_manager.cc:1398] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:02.706506 9879 consensus_queue.cc:260] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.706784 9879 ts_tablet_manager.cc:1398] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:02.707074 9890 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:02.707877 9771 raft_consensus.cc:482] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:02.707984 9771 raft_consensus.cc:504] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.708155 9890 tablet_bootstrap.cc:654] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.708582 9771 leader_election.cc:288] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:02.708766 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188101b7f3784e48975a673952b2bc63" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:02.708789 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188101b7f3784e48975a673952b2bc63" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:09:02.708885 9015 raft_consensus.cc:2454] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:02.709040 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188101b7f3784e48975a673952b2bc63" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:02.709115 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188101b7f3784e48975a673952b2bc63" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:02.709173 8885 raft_consensus.cc:2454] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:02.709242 8754 raft_consensus.cc:2454] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:02.709481 9211 leader_election.cc:302] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 84c7d91c82524b69a9184fcf1949b5b0, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
W20240501 14:09:02.709715 9211 leader_election.cc:341] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:09:02.709883 9890 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:02.709962 9890 ts_tablet_manager.cc:1367] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:02.710137 9771 raft_consensus.cc:2781] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:02.710192 9771 raft_consensus.cc:482] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:02.710227 9771 raft_consensus.cc:3037] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.710379 9890 raft_consensus.cc:348] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.710490 9890 raft_consensus.cc:374] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.710520 9890 raft_consensus.cc:729] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.710642 9890 consensus_queue.cc:260] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.710950 9890 ts_tablet_manager.cc:1398] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:02.711366 9890 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:02.712256 9890 tablet_bootstrap.cc:654] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:02.713343 9890 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:02.713414 9890 ts_tablet_manager.cc:1367] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:02.713809 9890 raft_consensus.cc:348] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.713920 9890 raft_consensus.cc:374] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:02.713959 9890 raft_consensus.cc:729] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:02.714087 9890 consensus_queue.cc:260] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.714362 9890 ts_tablet_manager.cc:1398] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240501 14:09:02.739969 9081 consensus_peers.cc:485] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Status: Network error: Client connection negotiation failed: client connection to 127.6.153.199:42105: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240501 14:09:02.779198 9742 raft_consensus.cc:482] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:02.779359 9742 raft_consensus.cc:504] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.779891 9742 leader_election.cc:288] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:02.780220 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72446fcffed439c8653937330e5b8ee" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:02.780277 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72446fcffed439c8653937330e5b8ee" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:09:02.780364 8754 raft_consensus.cc:2454] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:02.780336 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72446fcffed439c8653937330e5b8ee" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:02.780398 9277 raft_consensus.cc:2454] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:02.780472 9409 raft_consensus.cc:2454] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:02.780637 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72446fcffed439c8653937330e5b8ee" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:02.780735 8949 leader_election.cc:302] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 25737230978048598f426b1a99b27441, 6b961a167ae64dbc9d102ce5ddf721f3, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:02.780783 9147 raft_consensus.cc:2454] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:02.780894 9742 raft_consensus.cc:2781] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:02.780937 9742 raft_consensus.cc:482] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:02.780968 9742 raft_consensus.cc:3037] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.794729 9742 raft_consensus.cc:504] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.794728 9771 raft_consensus.cc:504] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.795306 9771 leader_election.cc:288] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 election: Requested vote from peers 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:02.795318 9742 leader_election.cc:288] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 election: Requested vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:02.795881 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188101b7f3784e48975a673952b2bc63" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:02.795845 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188101b7f3784e48975a673952b2bc63" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:02.795938 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188101b7f3784e48975a673952b2bc63" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:02.796001 9409 raft_consensus.cc:3037] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.796015 8754 raft_consensus.cc:3037] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.796031 9015 raft_consensus.cc:3037] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.796948 8754 raft_consensus.cc:2454] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:02.796944 9015 raft_consensus.cc:2454] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:02.796953 9409 raft_consensus.cc:2454] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:02.797314 9212 leader_election.cc:302] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 25737230978048598f426b1a99b27441, 6b961a167ae64dbc9d102ce5ddf721f3; no voters:
I20240501 14:09:02.797473 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72446fcffed439c8653937330e5b8ee" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:02.797602 9277 raft_consensus.cc:3037] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.797636 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188101b7f3784e48975a673952b2bc63" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:02.797755 8885 raft_consensus.cc:3037] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.798238 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72446fcffed439c8653937330e5b8ee" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:02.798383 9147 raft_consensus.cc:3037] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.798538 9277 raft_consensus.cc:2454] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:02.797513 9771 raft_consensus.cc:2781] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:02.798643 8885 raft_consensus.cc:2454] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:02.798851 9771 raft_consensus.cc:686] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Becoming Leader. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Running, Role: LEADER
I20240501 14:09:02.798867 8752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72446fcffed439c8653937330e5b8ee" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:02.798985 8752 raft_consensus.cc:3037] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.799022 9771 consensus_queue.cc:237] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:02.799176 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72446fcffed439c8653937330e5b8ee" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:02.799283 9409 raft_consensus.cc:3037] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.801257 8603 catalog_manager.cc:5484] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197). New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:02.807502 9851 raft_consensus.cc:482] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:02.807617 9851 raft_consensus.cc:504] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.808087 9851 leader_election.cc:288] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:02.808383 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21fe5dd65fe4fbea85fecc8ce19bc43" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:09:02.808425 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21fe5dd65fe4fbea85fecc8ce19bc43" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:02.808437 9408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21fe5dd65fe4fbea85fecc8ce19bc43" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:02.808471 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21fe5dd65fe4fbea85fecc8ce19bc43" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:02.808521 9015 raft_consensus.cc:2454] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.808564 8885 raft_consensus.cc:2454] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.808573 9408 raft_consensus.cc:2454] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.808591 9145 raft_consensus.cc:2454] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 0.
I20240501 14:09:02.808859 8687 leader_election.cc:302] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 25737230978048598f426b1a99b27441, 84c7d91c82524b69a9184fcf1949b5b0, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:02.809024 9851 raft_consensus.cc:2781] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:02.809084 9851 raft_consensus.cc:482] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:02.809109 9851 raft_consensus.cc:3037] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.888401 9147 raft_consensus.cc:2454] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:02.888401 9409 raft_consensus.cc:2454] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:02.888401 8752 raft_consensus.cc:2454] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:02.888911 8949 leader_election.cc:302] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 6b961a167ae64dbc9d102ce5ddf721f3, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:02.889165 9742 raft_consensus.cc:2781] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:02.889343 9742 raft_consensus.cc:686] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Becoming Leader. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Running, Role: LEADER
I20240501 14:09:02.889515 9742 consensus_queue.cc:237] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.889665 9851 raft_consensus.cc:504] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.890165 9851 leader_election.cc:288] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 election: Requested vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:02.890614 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21fe5dd65fe4fbea85fecc8ce19bc43" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:02.890736 9147 raft_consensus.cc:3037] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.891075 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21fe5dd65fe4fbea85fecc8ce19bc43" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:02.891182 9015 raft_consensus.cc:3037] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.891382 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21fe5dd65fe4fbea85fecc8ce19bc43" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:02.891492 9409 raft_consensus.cc:3037] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.891929 9147 raft_consensus.cc:2454] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.892071 9015 raft_consensus.cc:2454] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.892383 9409 raft_consensus.cc:2454] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
I20240501 14:09:02.892477 8690 leader_election.cc:302] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 25737230978048598f426b1a99b27441, 73ea0b7a5e4c45a3814d46e74d62331a, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:02.892707 9851 raft_consensus.cc:2781] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:02.892681 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21fe5dd65fe4fbea85fecc8ce19bc43" candidate_uuid: "25737230978048598f426b1a99b27441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:02.892794 9851 raft_consensus.cc:686] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Becoming Leader. State: Replica: 25737230978048598f426b1a99b27441, State: Running, Role: LEADER
I20240501 14:09:02.892800 8885 raft_consensus.cc:3037] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:02.892930 9851 consensus_queue.cc:237] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:02.893007 8603 catalog_manager.cc:5484] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:02.894649 8604 catalog_manager.cc:5484] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25737230978048598f426b1a99b27441 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:02.981333 8885 raft_consensus.cc:2454] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25737230978048598f426b1a99b27441 in term 1.
W20240501 14:09:03.026325 9905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:03.026592 9905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:03.029779 9905 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 14:09:03.121744 9277 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.121775 9409 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.122267 9147 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.122268 9902 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.122400 9015 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.122933 9742 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.123080 8885 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.123469 9851 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.123766 9851 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.123831 9408 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.127512 8752 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.127947 9850 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.128278 9850 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.128269 9742 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.128837 9145 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.129565 9902 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.134465 9408 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.134790 9016 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.135846 8754 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.136050 8883 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.136399 9771 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.136708 9771 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.137406 9900 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.138067 9900 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 14:09:03.174080 9081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.153.199:42105: connect: Connection refused (error 111) [suppressed 10 similar messages]
INSERT report
rows total: 2000
time total: 178.322 ms
time per row: 0.0891611 ms
I20240501 14:09:03.250746 8609 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40168:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:03.256390 8996 tablet_service.cc:1467] Processing CreateTablet for tablet 7eac39a2ecdc40f0929ab332508189cd (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.256543 9258 tablet_service.cc:1467] Processing CreateTablet for tablet 7eac39a2ecdc40f0929ab332508189cd (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.256698 8865 tablet_service.cc:1467] Processing CreateTablet for tablet 7eac39a2ecdc40f0929ab332508189cd (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.256758 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac39a2ecdc40f0929ab332508189cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.256830 9258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac39a2ecdc40f0929ab332508189cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.256938 8865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac39a2ecdc40f0929ab332508189cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.257675 9127 tablet_service.cc:1467] Processing CreateTablet for tablet 7eac39a2ecdc40f0929ab332508189cd (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.257942 9127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac39a2ecdc40f0929ab332508189cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.259455 8994 tablet_service.cc:1467] Processing CreateTablet for tablet 203c4d95b91f495f91f96b9308d1efc3 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.259475 8732 tablet_service.cc:1467] Processing CreateTablet for tablet 203c4d95b91f495f91f96b9308d1efc3 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.259703 8994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 203c4d95b91f495f91f96b9308d1efc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.259732 8732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 203c4d95b91f495f91f96b9308d1efc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.260092 9389 tablet_service.cc:1467] Processing CreateTablet for tablet 2d86cde53ceb4b63842ea9ec09659318 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.260346 9389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d86cde53ceb4b63842ea9ec09659318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.260483 9383 tablet_service.cc:1467] Processing CreateTablet for tablet 7eac39a2ecdc40f0929ab332508189cd (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.260704 9383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac39a2ecdc40f0929ab332508189cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.262688 8996 tablet_service.cc:1467] Processing CreateTablet for tablet 2d86cde53ceb4b63842ea9ec09659318 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.262940 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d86cde53ceb4b63842ea9ec09659318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.262923 8863 tablet_service.cc:1467] Processing CreateTablet for tablet 203c4d95b91f495f91f96b9308d1efc3 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.263085 8865 tablet_service.cc:1467] Processing CreateTablet for tablet 2d86cde53ceb4b63842ea9ec09659318 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.263096 9126 tablet_service.cc:1467] Processing CreateTablet for tablet 203c4d95b91f495f91f96b9308d1efc3 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.263157 8863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 203c4d95b91f495f91f96b9308d1efc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.263309 8865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d86cde53ceb4b63842ea9ec09659318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.263309 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 203c4d95b91f495f91f96b9308d1efc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.265065 9127 tablet_service.cc:1467] Processing CreateTablet for tablet 2d86cde53ceb4b63842ea9ec09659318 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.265314 9127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d86cde53ceb4b63842ea9ec09659318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.265441 9981 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:03.265709 9248 tablet_service.cc:1467] Processing CreateTablet for tablet 203c4d95b91f495f91f96b9308d1efc3 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.265899 9985 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:03.265949 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 203c4d95b91f495f91f96b9308d1efc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.266340 9981 tablet_bootstrap.cc:654] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.266454 9986 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:03.267009 9982 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:03.267419 9983 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:03.265723 9258 tablet_service.cc:1467] Processing CreateTablet for tablet 2d86cde53ceb4b63842ea9ec09659318 (DEFAULT_TABLE table=rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.267738 9258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d86cde53ceb4b63842ea9ec09659318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.267819 9985 tablet_bootstrap.cc:654] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.267944 9982 tablet_bootstrap.cc:654] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.268527 9983 tablet_bootstrap.cc:654] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.268666 9986 tablet_bootstrap.cc:654] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.269023 9981 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:03.269119 9981 ts_tablet_manager.cc:1367] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:03.269582 9981 raft_consensus.cc:348] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.269702 9981 raft_consensus.cc:374] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.269733 9981 raft_consensus.cc:729] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.269794 9984 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:03.269878 9981 consensus_queue.cc:260] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.270161 9981 ts_tablet_manager.cc:1398] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.270318 9981 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:03.271157 9981 tablet_bootstrap.cc:654] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.271497 9984 tablet_bootstrap.cc:654] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.271675 9983 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:03.271756 9983 ts_tablet_manager.cc:1367] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:03.271929 9982 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:03.272013 9982 ts_tablet_manager.cc:1367] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 14:09:03.272208 9983 raft_consensus.cc:348] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.272325 9983 raft_consensus.cc:374] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.272357 9983 raft_consensus.cc:729] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.272480 9983 consensus_queue.cc:260] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.272497 9982 raft_consensus.cc:348] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.272665 9982 raft_consensus.cc:374] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.272703 9982 raft_consensus.cc:729] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.272750 9983 ts_tablet_manager.cc:1398] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.272819 9982 consensus_queue.cc:260] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.272929 9983 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:03.273073 9982 ts_tablet_manager.cc:1398] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.273372 9982 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:03.273423 9985 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:03.273509 9985 ts_tablet_manager.cc:1367] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 14:09:03.273772 9983 tablet_bootstrap.cc:654] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.273942 9985 raft_consensus.cc:348] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.274049 9985 raft_consensus.cc:374] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.274091 9985 raft_consensus.cc:729] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.274191 9982 tablet_bootstrap.cc:654] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.274204 9985 consensus_queue.cc:260] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.274564 9985 ts_tablet_manager.cc:1398] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.275350 9986 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:03.275451 9986 ts_tablet_manager.cc:1367] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240501 14:09:03.276353 9981 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:03.276439 9981 ts_tablet_manager.cc:1367] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:09:03.276582 9986 raft_consensus.cc:348] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.276692 9986 raft_consensus.cc:374] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.276723 9986 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.276844 9986 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.276885 9981 raft_consensus.cc:348] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.276978 9981 raft_consensus.cc:374] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.277005 9981 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.277575 9986 ts_tablet_manager.cc:1398] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:03.277751 9986 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:03.277694 9981 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.277978 9981 ts_tablet_manager.cc:1398] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.278126 9981 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:03.278735 9986 tablet_bootstrap.cc:654] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.279139 9981 tablet_bootstrap.cc:654] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.279875 9984 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:03.279980 9984 ts_tablet_manager.cc:1367] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240501 14:09:03.280150 9986 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:03.280222 9986 ts_tablet_manager.cc:1367] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:09:03.280443 9984 raft_consensus.cc:348] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.280565 9984 raft_consensus.cc:374] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.280604 9984 raft_consensus.cc:729] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.280622 9986 raft_consensus.cc:348] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.280716 9986 raft_consensus.cc:374] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.280750 9986 raft_consensus.cc:729] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.280728 9984 consensus_queue.cc:260] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.280861 9986 consensus_queue.cc:260] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.281102 9986 ts_tablet_manager.cc:1398] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.281150 9984 ts_tablet_manager.cc:1398] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:03.281304 9984 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:03.281714 9981 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:03.281782 9981 ts_tablet_manager.cc:1367] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 14:09:03.282210 9984 tablet_bootstrap.cc:654] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.282155 9981 raft_consensus.cc:348] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.282250 9981 raft_consensus.cc:374] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.282277 9981 raft_consensus.cc:729] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.282389 9981 consensus_queue.cc:260] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.282662 9981 ts_tablet_manager.cc:1398] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:03.283907 9983 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:03.284008 9983 ts_tablet_manager.cc:1367] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240501 14:09:03.284430 9983 raft_consensus.cc:348] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.284548 9983 raft_consensus.cc:374] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.284585 9983 raft_consensus.cc:729] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.284705 9983 consensus_queue.cc:260] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.285286 9982 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:03.285383 9982 ts_tablet_manager.cc:1367] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 14:09:03.285408 9984 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:03.285471 9984 ts_tablet_manager.cc:1367] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 14:09:03.285562 9983 ts_tablet_manager.cc:1398] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:03.285694 9983 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:03.285835 9982 raft_consensus.cc:348] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.285835 9984 raft_consensus.cc:348] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.285938 9984 raft_consensus.cc:374] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.285938 9982 raft_consensus.cc:374] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.285969 9984 raft_consensus.cc:729] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.285969 9982 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.286087 9982 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.286094 9984 consensus_queue.cc:260] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.286346 9984 ts_tablet_manager.cc:1398] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.286346 9982 ts_tablet_manager.cc:1398] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.286409 9983 tablet_bootstrap.cc:654] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.286501 9982 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:03.286505 9984 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:03.287305 9982 tablet_bootstrap.cc:654] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.287305 9984 tablet_bootstrap.cc:654] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.287997 9983 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:03.288084 9983 ts_tablet_manager.cc:1367] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:03.288477 9983 raft_consensus.cc:348] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.288583 9983 raft_consensus.cc:374] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.288614 9983 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.288722 9983 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.289000 9983 ts_tablet_manager.cc:1398] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.289557 9984 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:03.289551 9982 tablet_bootstrap.cc:492] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:03.289638 9984 ts_tablet_manager.cc:1367] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:03.289649 9982 ts_tablet_manager.cc:1367] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:09:03.290027 9982 raft_consensus.cc:348] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.290027 9984 raft_consensus.cc:348] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.290132 9982 raft_consensus.cc:374] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.290144 9984 raft_consensus.cc:374] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.290163 9982 raft_consensus.cc:729] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.290174 9984 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.290282 9984 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.290284 9982 consensus_queue.cc:260] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.290544 9982 ts_tablet_manager.cc:1398] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.290544 9984 ts_tablet_manager.cc:1398] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.312673 9759 raft_consensus.cc:482] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:03.312848 9759 raft_consensus.cc:504] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.314145 9759 leader_election.cc:288] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983)
I20240501 14:09:03.319635 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203c4d95b91f495f91f96b9308d1efc3" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:09:03.319725 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203c4d95b91f495f91f96b9308d1efc3" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:03.319818 9277 raft_consensus.cc:2454] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 0.
I20240501 14:09:03.319875 9145 raft_consensus.cc:2454] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 0.
I20240501 14:09:03.319911 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203c4d95b91f495f91f96b9308d1efc3" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:09:03.319989 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203c4d95b91f495f91f96b9308d1efc3" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:03.320026 9016 raft_consensus.cc:2454] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 0.
I20240501 14:09:03.320115 8754 raft_consensus.cc:2454] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 0.
I20240501 14:09:03.320261 8820 leader_election.cc:302] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 73ea0b7a5e4c45a3814d46e74d62331a, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:03.320545 9759 raft_consensus.cc:2781] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:03.320611 9759 raft_consensus.cc:482] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:03.320648 9759 raft_consensus.cc:3037] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.321549 9759 raft_consensus.cc:504] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.321997 9759 leader_election.cc:288] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 1 election: Requested vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983)
I20240501 14:09:03.322228 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203c4d95b91f495f91f96b9308d1efc3" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:03.322268 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203c4d95b91f495f91f96b9308d1efc3" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:03.322297 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203c4d95b91f495f91f96b9308d1efc3" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:03.322346 8754 raft_consensus.cc:3037] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.322324 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203c4d95b91f495f91f96b9308d1efc3" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:03.322361 9277 raft_consensus.cc:3037] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.322407 9145 raft_consensus.cc:3037] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.322419 9016 raft_consensus.cc:3037] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.323307 8754 raft_consensus.cc:2454] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 1.
I20240501 14:09:03.323307 9277 raft_consensus.cc:2454] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 1.
I20240501 14:09:03.323418 9145 raft_consensus.cc:2454] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 1.
I20240501 14:09:03.323576 9016 raft_consensus.cc:2454] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 1.
I20240501 14:09:03.323635 8820 leader_election.cc:302] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 25737230978048598f426b1a99b27441, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:03.323791 9759 raft_consensus.cc:2781] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:03.323994 9759 raft_consensus.cc:686] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 LEADER]: Becoming Leader. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Running, Role: LEADER
I20240501 14:09:03.324316 9759 consensus_queue.cc:237] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.326834 8603 catalog_manager.cc:5484] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194). New cstate: current_term: 1 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:03.338330 9948 raft_consensus.cc:482] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:03.338485 9948 raft_consensus.cc:504] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.339193 9948 leader_election.cc:288] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:03.339512 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac39a2ecdc40f0929ab332508189cd" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:09:03.339653 9016 raft_consensus.cc:2454] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:03.340292 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac39a2ecdc40f0929ab332508189cd" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:03.340435 8885 raft_consensus.cc:2454] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:03.340399 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac39a2ecdc40f0929ab332508189cd" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:03.340538 9409 raft_consensus.cc:2454] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:03.340749 9211 leader_election.cc:302] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 84c7d91c82524b69a9184fcf1949b5b0, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:03.341015 9948 raft_consensus.cc:2781] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:03.341120 9948 raft_consensus.cc:482] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:03.341270 9948 raft_consensus.cc:3037] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.342576 9948 raft_consensus.cc:504] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.343078 9948 leader_election.cc:288] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 election: Requested vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:03.343413 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac39a2ecdc40f0929ab332508189cd" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:03.343475 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac39a2ecdc40f0929ab332508189cd" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:03.343538 9016 raft_consensus.cc:3037] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.343603 9409 raft_consensus.cc:3037] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.343605 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac39a2ecdc40f0929ab332508189cd" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:03.343720 8885 raft_consensus.cc:3037] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.344511 9016 raft_consensus.cc:2454] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:03.344552 9409 raft_consensus.cc:2454] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:03.344552 8885 raft_consensus.cc:2454] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:03.344873 9211 leader_election.cc:302] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 6b961a167ae64dbc9d102ce5ddf721f3, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:03.345021 9948 raft_consensus.cc:2781] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:03.345096 9948 raft_consensus.cc:686] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Becoming Leader. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Running, Role: LEADER
I20240501 14:09:03.345237 9948 consensus_queue.cc:237] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.345808 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac39a2ecdc40f0929ab332508189cd" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:03.345808 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac39a2ecdc40f0929ab332508189cd" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:03.345948 9145 raft_consensus.cc:2454] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:03.347165 8603 catalog_manager.cc:5484] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197). New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:03.389669 9948 raft_consensus.cc:482] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:03.389794 9948 raft_consensus.cc:504] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.390245 9948 leader_election.cc:288] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:03.390542 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:09:03.390601 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:03.390616 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:03.390686 9016 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:03.390712 9409 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:03.390766 9145 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:03.390781 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:03.390904 8885 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 0.
I20240501 14:09:03.390986 9211 leader_election.cc:302] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 6b961a167ae64dbc9d102ce5ddf721f3, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:03.391178 9948 raft_consensus.cc:2781] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:03.391237 9948 raft_consensus.cc:482] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:03.391263 9948 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.392091 9948 raft_consensus.cc:504] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.392693 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:03.392757 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:03.392796 9016 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.392834 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:03.392899 9145 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.392930 9409 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.393651 9016 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:03.393678 9145 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:03.393738 9409 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:03.393975 9211 leader_election.cc:302] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 4 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 6b961a167ae64dbc9d102ce5ddf721f3, 73ea0b7a5e4c45a3814d46e74d62331a, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:03.394060 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "0c0d3d3aae054876a0a522a8325ddd02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:03.394141 9947 raft_consensus.cc:2781] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:03.394184 8885 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.394198 9947 raft_consensus.cc:686] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Becoming Leader. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Running, Role: LEADER
I20240501 14:09:03.394335 9947 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } }
I20240501 14:09:03.392485 9948 leader_election.cc:288] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [CANDIDATE]: Term 1 election: Requested vote from peers 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:03.395087 8885 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c0d3d3aae054876a0a522a8325ddd02 in term 1.
I20240501 14:09:03.396231 8603 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197). New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:03.511751 10012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:03.512023 10012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:03.515697 10012 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 14:09:03.590418 8885 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.590965 9771 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.591567 9016 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.592027 9145 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.592466 9771 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.592571 8885 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.593295 9408 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.593295 9409 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.595118 9947 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.595444 9947 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.595666 9947 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.595911 9947 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.601182 9015 raft_consensus.cc:1261] T 203c4d95b91f495f91f96b9308d1efc3 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.601972 8754 raft_consensus.cc:1261] T 203c4d95b91f495f91f96b9308d1efc3 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.602418 9277 raft_consensus.cc:1261] T 203c4d95b91f495f91f96b9308d1efc3 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.602773 9759 consensus_queue.cc:1035] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.604271 9759 consensus_queue.cc:1035] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.605718 9147 raft_consensus.cc:1261] T 203c4d95b91f495f91f96b9308d1efc3 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.608232 9759 consensus_queue.cc:1035] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.608492 9759 consensus_queue.cc:1035] T 203c4d95b91f495f91f96b9308d1efc3 P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.649855 9015 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.650274 9947 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.652829 9146 raft_consensus.cc:3037] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.655568 9146 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:03.658624 9947 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:03.664848 9945 mvcc.cc:205] Tried to move back new op lower bound from 7022889138541735936 to 7022889137543909376. Current Snapshot: MvccSnapshot[applied={T|T < 7022889138541735936}]
INSERT report
rows total: 2000
time total: 146.934 ms
time per row: 0.073467 ms
I20240501 14:09:03.711761 8612 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40168:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:03.717314 8732 tablet_service.cc:1467] Processing CreateTablet for tablet 858bf448fd5249f0a86d17d6d011a6b4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.717343 8996 tablet_service.cc:1467] Processing CreateTablet for tablet 858bf448fd5249f0a86d17d6d011a6b4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.717619 8732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858bf448fd5249f0a86d17d6d011a6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.717615 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858bf448fd5249f0a86d17d6d011a6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.717773 9383 tablet_service.cc:1467] Processing CreateTablet for tablet 858bf448fd5249f0a86d17d6d011a6b4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.718052 9383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858bf448fd5249f0a86d17d6d011a6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.719653 8863 tablet_service.cc:1467] Processing CreateTablet for tablet 858bf448fd5249f0a86d17d6d011a6b4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.719861 9986 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:03.719944 8863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858bf448fd5249f0a86d17d6d011a6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.719960 9985 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:03.720628 8865 tablet_service.cc:1467] Processing CreateTablet for tablet 30ef391ceec44fd4b7195b41d7732843 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.720688 9986 tablet_bootstrap.cc:654] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.720770 9985 tablet_bootstrap.cc:654] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.720870 8865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef391ceec44fd4b7195b41d7732843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.721942 9982 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:03.722626 8864 tablet_service.cc:1467] Processing CreateTablet for tablet 96ab0c0bda54481b95b0fdf9d8da84df (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.722775 9982 tablet_bootstrap.cc:654] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.722843 8864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ab0c0bda54481b95b0fdf9d8da84df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.723158 8994 tablet_service.cc:1467] Processing CreateTablet for tablet 30ef391ceec44fd4b7195b41d7732843 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.723201 9389 tablet_service.cc:1467] Processing CreateTablet for tablet 96ab0c0bda54481b95b0fdf9d8da84df (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.723436 8994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef391ceec44fd4b7195b41d7732843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.723462 9389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ab0c0bda54481b95b0fdf9d8da84df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.725060 8734 tablet_service.cc:1467] Processing CreateTablet for tablet 96ab0c0bda54481b95b0fdf9d8da84df (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.725289 8734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ab0c0bda54481b95b0fdf9d8da84df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.725410 9983 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:03.725734 9383 tablet_service.cc:1467] Processing CreateTablet for tablet 30ef391ceec44fd4b7195b41d7732843 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.725946 9383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef391ceec44fd4b7195b41d7732843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.726209 9983 tablet_bootstrap.cc:654] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.727540 8732 tablet_service.cc:1467] Processing CreateTablet for tablet 30ef391ceec44fd4b7195b41d7732843 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.727649 9127 tablet_service.cc:1467] Processing CreateTablet for tablet 858bf448fd5249f0a86d17d6d011a6b4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.727759 8732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef391ceec44fd4b7195b41d7732843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.727907 9127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858bf448fd5249f0a86d17d6d011a6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.727969 9258 tablet_service.cc:1467] Processing CreateTablet for tablet 96ab0c0bda54481b95b0fdf9d8da84df (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.728231 9258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ab0c0bda54481b95b0fdf9d8da84df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.729722 9248 tablet_service.cc:1467] Processing CreateTablet for tablet 30ef391ceec44fd4b7195b41d7732843 (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.729784 9984 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:03.729948 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef391ceec44fd4b7195b41d7732843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.729995 9126 tablet_service.cc:1467] Processing CreateTablet for tablet 96ab0c0bda54481b95b0fdf9d8da84df (DEFAULT_TABLE table=rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:03.730194 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ab0c0bda54481b95b0fdf9d8da84df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:03.730651 9983 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:03.730744 9983 ts_tablet_manager.cc:1367] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:09:03.731189 9983 raft_consensus.cc:348] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.731325 9983 raft_consensus.cc:374] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.731386 9983 raft_consensus.cc:729] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.731508 9983 consensus_queue.cc:260] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.731807 9983 ts_tablet_manager.cc:1398] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.732003 9981 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:03.732784 9981 tablet_bootstrap.cc:654] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.732966 9983 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:03.734098 9983 tablet_bootstrap.cc:654] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.735108 9983 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0: No bootstrap required, opened a new log
I20240501 14:09:03.735185 9983 ts_tablet_manager.cc:1367] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:03.735177 9982 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:03.735271 9982 ts_tablet_manager.cc:1367] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240501 14:09:03.735630 9983 raft_consensus.cc:348] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.735731 9983 raft_consensus.cc:374] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.735760 9983 raft_consensus.cc:729] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.735837 9985 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:03.735868 9983 consensus_queue.cc:260] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.735920 9985 ts_tablet_manager.cc:1367] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240501 14:09:03.735992 9981 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:03.735950 9982 raft_consensus.cc:348] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.736061 9981 ts_tablet_manager.cc:1367] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 14:09:03.736069 9982 raft_consensus.cc:374] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.736099 9982 raft_consensus.cc:729] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.736145 9983 ts_tablet_manager.cc:1398] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.736212 9982 consensus_queue.cc:260] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.736321 9985 raft_consensus.cc:348] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.736419 9985 raft_consensus.cc:374] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.736454 9985 raft_consensus.cc:729] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.736472 9982 ts_tablet_manager.cc:1398] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:03.736465 9981 raft_consensus.cc:348] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.736569 9981 raft_consensus.cc:374] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.736610 9981 raft_consensus.cc:729] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.736575 9985 consensus_queue.cc:260] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.736629 9982 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:03.736732 9981 consensus_queue.cc:260] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.736850 9985 ts_tablet_manager.cc:1398] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.736991 9981 ts_tablet_manager.cc:1398] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:03.737021 9985 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:03.737143 9981 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02: Bootstrap starting.
I20240501 14:09:03.737454 9982 tablet_bootstrap.cc:654] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.737798 9985 tablet_bootstrap.cc:654] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.737946 9981 tablet_bootstrap.cc:654] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.740177 9982 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:03.740262 9982 ts_tablet_manager.cc:1367] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:09:03.740659 9982 raft_consensus.cc:348] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.740769 9982 raft_consensus.cc:374] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.740803 9982 raft_consensus.cc:729] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.740914 9982 consensus_queue.cc:260] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.741197 9982 ts_tablet_manager.cc:1398] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.741346 9982 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: Bootstrap starting.
I20240501 14:09:03.742168 9982 tablet_bootstrap.cc:654] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.742578 9981 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02: No bootstrap required, opened a new log
I20240501 14:09:03.742707 9981 ts_tablet_manager.cc:1367] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240501 14:09:03.743189 9981 raft_consensus.cc:348] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.743296 9981 raft_consensus.cc:374] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.743352 9981 raft_consensus.cc:729] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.743485 9981 consensus_queue.cc:260] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.743649 9982 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: No bootstrap required, opened a new log
I20240501 14:09:03.743721 9982 ts_tablet_manager.cc:1367] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:03.744138 9984 tablet_bootstrap.cc:654] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.744108 9982 raft_consensus.cc:348] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.744184 9982 raft_consensus.cc:374] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.744205 9982 raft_consensus.cc:729] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.744199 9981 ts_tablet_manager.cc:1398] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.744302 9982 consensus_queue.cc:260] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.744529 9982 ts_tablet_manager.cc:1398] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.745247 9984 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:03.745342 9984 ts_tablet_manager.cc:1367] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.001s
I20240501 14:09:03.745783 9984 raft_consensus.cc:348] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.745887 9984 raft_consensus.cc:374] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.745918 9984 raft_consensus.cc:729] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.746038 9984 consensus_queue.cc:260] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.746290 9984 ts_tablet_manager.cc:1398] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:03.746464 9984 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: Bootstrap starting.
I20240501 14:09:03.747224 9984 tablet_bootstrap.cc:654] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.748476 9984 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: No bootstrap required, opened a new log
I20240501 14:09:03.748564 9984 ts_tablet_manager.cc:1367] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:03.748980 9984 raft_consensus.cc:348] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.749087 9984 raft_consensus.cc:374] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.749117 9984 raft_consensus.cc:729] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.749243 9984 consensus_queue.cc:260] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.749511 9984 ts_tablet_manager.cc:1398] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.755744 9986 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:03.755859 9986 ts_tablet_manager.cc:1367] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.000s
I20240501 14:09:03.756304 9986 raft_consensus.cc:348] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.756414 9986 raft_consensus.cc:374] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.756446 9986 raft_consensus.cc:729] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.756577 9986 consensus_queue.cc:260] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:03.756807 9986 ts_tablet_manager.cc:1398] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.756966 9986 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:03.757918 9986 tablet_bootstrap.cc:654] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.758913 9986 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:03.759007 9986 ts_tablet_manager.cc:1367] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:03.759368 9986 raft_consensus.cc:348] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.759474 9986 raft_consensus.cc:374] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.759505 9986 raft_consensus.cc:729] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.759627 9986 consensus_queue.cc:260] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.759865 9986 ts_tablet_manager.cc:1398] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.760049 9986 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3: Bootstrap starting.
I20240501 14:09:03.761056 9986 tablet_bootstrap.cc:654] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.761915 9986 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3: No bootstrap required, opened a new log
I20240501 14:09:03.761989 9986 ts_tablet_manager.cc:1367] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:03.762338 9986 raft_consensus.cc:348] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.762424 9986 raft_consensus.cc:374] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.762444 9986 raft_consensus.cc:729] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b961a167ae64dbc9d102ce5ddf721f3, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.762565 9986 consensus_queue.cc:260] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.762822 9986 ts_tablet_manager.cc:1398] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.763664 9985 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:03.763753 9985 ts_tablet_manager.cc:1367] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20240501 14:09:03.764267 9985 raft_consensus.cc:348] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.764387 9985 raft_consensus.cc:374] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.764425 9985 raft_consensus.cc:729] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.764554 9985 consensus_queue.cc:260] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.764849 9985 ts_tablet_manager.cc:1398] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:03.765086 9985 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:03.765940 9985 tablet_bootstrap.cc:654] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:03.766745 9977 raft_consensus.cc:482] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:03.766846 9977 raft_consensus.cc:504] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.767247 9985 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441: No bootstrap required, opened a new log
I20240501 14:09:03.767356 9985 ts_tablet_manager.cc:1367] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:03.767453 9977 leader_election.cc:288] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:03.767902 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef391ceec44fd4b7195b41d7732843" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:03.767762 9985 raft_consensus.cc:348] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.768029 9985 raft_consensus.cc:374] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:03.768065 9985 raft_consensus.cc:729] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: FOLLOWER
I20240501 14:09:03.768054 8885 raft_consensus.cc:2454] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:03.768114 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef391ceec44fd4b7195b41d7732843" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:03.768129 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef391ceec44fd4b7195b41d7732843" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:09:03.768168 9985 consensus_queue.cc:260] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.768241 9277 raft_consensus.cc:2454] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:03.768325 9408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef391ceec44fd4b7195b41d7732843" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:03.768410 9985 ts_tablet_manager.cc:1398] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:03.768445 9408 raft_consensus.cc:2454] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:03.768658 8754 raft_consensus.cc:2454] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 0.
I20240501 14:09:03.768726 8949 leader_election.cc:302] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6b961a167ae64dbc9d102ce5ddf721f3, 84c7d91c82524b69a9184fcf1949b5b0, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:03.768958 9977 raft_consensus.cc:2781] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:03.769023 9977 raft_consensus.cc:482] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:03.769070 9977 raft_consensus.cc:3037] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.770061 9977 raft_consensus.cc:504] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.770531 9977 leader_election.cc:288] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 election: Requested vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:03.770682 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef391ceec44fd4b7195b41d7732843" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:03.770776 8754 raft_consensus.cc:3037] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.770747 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef391ceec44fd4b7195b41d7732843" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:03.770869 9277 raft_consensus.cc:3037] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.771742 8754 raft_consensus.cc:2454] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:03.771764 9277 raft_consensus.cc:2454] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:03.771967 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef391ceec44fd4b7195b41d7732843" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:03.772084 8885 raft_consensus.cc:3037] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.772966 8885 raft_consensus.cc:2454] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:03.773234 8949 leader_election.cc:302] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 84c7d91c82524b69a9184fcf1949b5b0, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:03.773396 9977 raft_consensus.cc:2781] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:03.773499 9977 raft_consensus.cc:686] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Becoming Leader. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Running, Role: LEADER
I20240501 14:09:03.773626 9977 consensus_queue.cc:237] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } }
I20240501 14:09:03.776170 9408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef391ceec44fd4b7195b41d7732843" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:03.776283 9408 raft_consensus.cc:3037] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.776599 8603 catalog_manager.cc:5484] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:03.777124 9408 raft_consensus.cc:2454] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 1.
I20240501 14:09:03.797463 10042 raft_consensus.cc:482] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:03.797612 10042 raft_consensus.cc:504] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.798292 10042 leader_election.cc:288] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:03.798772 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02" is_pre_election: true
I20240501 14:09:03.798756 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" is_pre_election: true
I20240501 14:09:03.798900 9277 raft_consensus.cc:2454] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 0.
I20240501 14:09:03.798933 9147 raft_consensus.cc:2454] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 0.
I20240501 14:09:03.799228 8820 leader_election.cc:302] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 73ea0b7a5e4c45a3814d46e74d62331a, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:03.799628 10042 raft_consensus.cc:2781] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:03.799698 10042 raft_consensus.cc:482] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:03.799664 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:03.799747 10042 raft_consensus.cc:3037] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.799793 8754 raft_consensus.cc:2454] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 0.
I20240501 14:09:03.800822 10042 raft_consensus.cc:504] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.801338 10042 leader_election.cc:288] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 1 election: Requested vote from peers 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:03.801590 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:03.801728 8754 raft_consensus.cc:3037] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.801796 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:03.801906 9277 raft_consensus.cc:3037] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.802618 8754 raft_consensus.cc:2454] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 1.
I20240501 14:09:03.802618 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:03.802734 9147 raft_consensus.cc:3037] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:03.802846 9277 raft_consensus.cc:2454] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 1.
I20240501 14:09:03.803179 8820 leader_election.cc:302] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 25737230978048598f426b1a99b27441, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:03.803366 10042 raft_consensus.cc:2781] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:03.803445 10042 raft_consensus.cc:686] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 LEADER]: Becoming Leader. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Running, Role: LEADER
I20240501 14:09:03.803597 10042 consensus_queue.cc:237] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } }
I20240501 14:09:03.803730 9147 raft_consensus.cc:2454] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 1.
I20240501 14:09:03.804489 9408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:03.804497 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:03.804649 9408 raft_consensus.cc:2454] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 0.
I20240501 14:09:03.805871 8603 catalog_manager.cc:5484] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194). New cstate: current_term: 1 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:04.022732 9640 raft_consensus.cc:482] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:04.022894 9640 raft_consensus.cc:504] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:04.023576 9640 leader_election.cc:288] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:04.024197 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bf448fd5249f0a86d17d6d011a6b4" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0" is_pre_election: true
I20240501 14:09:04.024264 9408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bf448fd5249f0a86d17d6d011a6b4" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" is_pre_election: true
I20240501 14:09:04.024353 9015 raft_consensus.cc:2454] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 0.
I20240501 14:09:04.024421 9408 raft_consensus.cc:2454] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 0.
I20240501 14:09:04.024765 9080 leader_election.cc:302] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6b961a167ae64dbc9d102ce5ddf721f3, 73ea0b7a5e4c45a3814d46e74d62331a, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:04.024948 9640 raft_consensus.cc:2781] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:04.024998 9640 raft_consensus.cc:482] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:04.025035 9640 raft_consensus.cc:3037] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:04.026103 9640 raft_consensus.cc:504] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:04.026654 9640 leader_election.cc:288] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [CANDIDATE]: Term 1 election: Requested vote from peers 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:04.026895 9015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bf448fd5249f0a86d17d6d011a6b4" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:04.027016 9015 raft_consensus.cc:3037] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:04.027127 9408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bf448fd5249f0a86d17d6d011a6b4" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:04.027230 9408 raft_consensus.cc:3037] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:04.028043 9015 raft_consensus.cc:2454] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1.
I20240501 14:09:04.028174 9408 raft_consensus.cc:2454] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1.
I20240501 14:09:04.028515 9080 leader_election.cc:302] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6b961a167ae64dbc9d102ce5ddf721f3, 73ea0b7a5e4c45a3814d46e74d62331a, 84c7d91c82524b69a9184fcf1949b5b0; no voters:
I20240501 14:09:04.028694 9640 raft_consensus.cc:2781] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:04.028658 8883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bf448fd5249f0a86d17d6d011a6b4" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
I20240501 14:09:04.028656 8752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bf448fd5249f0a86d17d6d011a6b4" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:04.028780 8883 raft_consensus.cc:3037] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:04.028788 8752 raft_consensus.cc:3037] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:04.028832 9640 raft_consensus.cc:686] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 LEADER]: Becoming Leader. State: Replica: 73ea0b7a5e4c45a3814d46e74d62331a, State: Running, Role: LEADER
I20240501 14:09:04.028982 9640 consensus_queue.cc:237] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } }
I20240501 14:09:04.029407 8754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bf448fd5249f0a86d17d6d011a6b4" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25737230978048598f426b1a99b27441" is_pre_election: true
I20240501 14:09:04.029698 8883 raft_consensus.cc:2454] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1.
I20240501 14:09:04.029724 8752 raft_consensus.cc:2454] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1.
I20240501 14:09:04.029831 8885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bf448fd5249f0a86d17d6d011a6b4" candidate_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" is_pre_election: true
I20240501 14:09:04.029943 8885 raft_consensus.cc:2362] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 73ea0b7a5e4c45a3814d46e74d62331a in term 1. Re-sending same reply.
I20240501 14:09:04.031066 8603 catalog_manager.cc:5484] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a reported cstate change: term changed from 0 to 1, leader changed from <none> to 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196). New cstate: current_term: 1 leader_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240501 14:09:04.181777 9902 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.201772 9408 raft_consensus.cc:3037] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:04.203038 10042 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 14:09:04.210592 10078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:04.210824 10078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:04.213960 10078 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 14:09:04.217000 10042 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.220290 10042 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.221500 10042 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.224857 9902 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.228857 10103 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.229032 9977 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_4'
I20240501 14:09:04.306151 9015 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:04.306439 8754 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:04.306845 9407 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:04.307245 8885 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:04.306541 10076 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 14:09:04.307708 9640 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.308588 10076 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.310454 10076 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:04.350593 10090 mvcc.cc:205] Tried to move back new op lower bound from 7022889141420621824 to 7022889140344836096. Current Snapshot: MvccSnapshot[applied={T|T < 7022889141420621824}]
I20240501 14:09:04.351435 10098 mvcc.cc:205] Tried to move back new op lower bound from 7022889141420621824 to 7022889140344836096. Current Snapshot: MvccSnapshot[applied={T|T < 7022889141420621824}]
I20240501 14:09:04.359019 10101 mvcc.cc:205] Tried to move back new op lower bound from 7022889141420621824 to 7022889140344836096. Current Snapshot: MvccSnapshot[applied={T|T < 7022889141420621824}]
INSERT report
rows total: 2000
time total: 145.943 ms
time per row: 0.0729716 ms
I20240501 14:09:04.404716 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.199:42105
--local_ip_for_outbound_sockets=127.6.153.199
--tserver_master_addrs=127.6.153.254:42171
--webserver_port=43067
--webserver_interface=127.6.153.199
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:09:04.510283 10161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:04.510489 10161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:04.510531 10161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:04.513585 10161 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:04.513645 10161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:04.513725 10161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.199
I20240501 14:09:04.517412 10161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.199:42105
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.6.153.199
--webserver_port=43067
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.10161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.199
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:04.518324 10161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:04.519465 10161 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:04.528090 10167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:04.528430 10169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:04.528911 10161 server_base.cc:1034] running on GCE node
W20240501 14:09:04.529109 10166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:04.529506 10161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:04.530064 10161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:04.531209 10161 hybrid_clock.cc:648] HybridClock initialized: now 1714572544531190 us; error 31 us; skew 500 ppm
I20240501 14:09:04.532979 10161 webserver.cc:460] Webserver started at http://127.6.153.199:43067/ using document root <none> and password file <none>
I20240501 14:09:04.533527 10161 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:04.533601 10161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:04.536816 10161 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:09:04.538843 10175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:04.540009 10161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:04.540103 10161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "9a65600939334f6db8dcca7f07923ece"
format_stamp: "Formatted at 2024-05-01 14:08:57 on dist-test-slave-cmf0"
I20240501 14:09:04.540591 10161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:04.564422 10161 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:04.565068 10161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:04.565207 10161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:04.565727 10161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:04.566848 10182 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240501 14:09:04.569931 10161 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240501 14:09:04.570003 10161 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20240501 14:09:04.570058 10161 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240501 14:09:04.571530 10161 ts_tablet_manager.cc:590] Registered 1 tablets
I20240501 14:09:04.571574 10161 ts_tablet_manager.cc:569] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:04.571841 10182 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:04.581426 10161 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.199:42105
I20240501 14:09:04.581470 10289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.199:42105 every 8 connection(s)
I20240501 14:09:04.582427 10161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 14:09:04.584501 10182 log.cc:824] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:04.585861 10182 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:04.586167 10182 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:04.586974 10182 ts_tablet_manager.cc:1367] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20240501 14:09:04.589378 10182 raft_consensus.cc:348] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:09:04.589690 10182 raft_consensus.cc:729] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: FOLLOWER
I20240501 14:09:04.590132 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 10161
I20240501 14:09:04.590135 10182 consensus_queue.cc:260] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } }
I20240501 14:09:04.590284 6759 test_util.cc:268] Using random seed: 125190604
I20240501 14:09:04.590870 10290 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:09:04.590966 10182 ts_tablet_manager.cc:1398] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 14:09:04.591141 10290 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:04.591770 10290 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:09:04.596226 8603 ts_manager.cc:194] Re-registered known tserver with Master: 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105)
I20240501 14:09:04.597065 8603 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.199:52615
I20240501 14:09:04.599567 10290 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
I20240501 14:09:04.602512 6759 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240501 14:09:04.602718 6759 test_util.cc:268] Using random seed: 125203036
I20240501 14:09:04.611497 6759 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240501 14:09:04.611650 6759 test_util.cc:268] Using random seed: 125211970
I20240501 14:09:04.626681 6759 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240501 14:09:04.626844 6759 test_util.cc:268] Using random seed: 125227162
I20240501 14:09:04.649487 6759 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240501 14:09:04.649679 6759 test_util.cc:268] Using random seed: 125249996
I20240501 14:09:04.701925 6759 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20240501 14:09:05.902065 8996 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:06.137970 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:06.141094 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:06.158017 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:06.175954 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:06.226313 9248 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:06.247946 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:06.685003 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:06.711805 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
W20240501 14:09:06.718660 9211 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.198:32925, user_credentials={real_user=slave}} blocked reactor thread for 48963.8us
W20240501 14:09:06.720069 9213 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.196:34983, user_credentials={real_user=slave}} blocked reactor thread for 47815.1us
W20240501 14:09:06.720819 9214 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.195:33719, user_credentials={real_user=slave}} blocked reactor thread for 46978.8us
I20240501 14:09:06.740084 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:06.787875 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:06.819721 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:06.822968 9248 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:06.830766 8996 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:07.174412 8749 consensus_queue.cc:237] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 54, Committed index: 54, Last appended: 1.55, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:07.177282 9142 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240501 14:09:07.177881 9851 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 54, Time since last communication: 0.000s
I20240501 14:09:07.180922 9013 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240501 14:09:07.187147 8881 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240501 14:09:07.188715 9851 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 14:09:07.193918 9408 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240501 14:09:07.193984 9851 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 54, Time since last communication: 0.000s
I20240501 14:09:07.203168 10322 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 54, Time since last communication: 0.000s
I20240501 14:09:07.203716 10322 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.204640 8881 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.208155 8604 catalog_manager.cc:5484] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.208937 9142 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
W20240501 14:09:07.215788 8688 consensus_peers.cc:485] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d21fe5dd65fe4fbea85fecc8ce19bc43. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:07.221532 9013 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.235806 9407 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.289461 9276 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 1.41, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:07.292891 9145 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20240501 14:09:07.294104 9948 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20240501 14:09:07.296603 9016 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20240501 14:09:07.298534 10344 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20240501 14:09:07.305162 10367 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.42: config changed from index -1 to 42, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.307956 9145 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index -1 to 42, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
W20240501 14:09:07.310109 9212 consensus_peers.cc:485] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d86cde53ceb4b63842ea9ec09659318. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:07.320983 9016 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index -1 to 42, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.322119 8607 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index -1 to 42, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:07.324295 9409 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20240501 14:09:07.340301 10368 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20240501 14:09:07.342679 8885 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20240501 14:09:07.364131 9406 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index -1 to 42, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.377127 10345 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 40, Time since last communication: 0.000s
I20240501 14:09:07.387275 9144 consensus_queue.cc:237] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 1.40, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:07.388839 8881 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index -1 to 42, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
W20240501 14:09:07.393673 9081 consensus_peers.cc:485] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 858bf448fd5249f0a86d17d6d011a6b4. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:07.395999 9407 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 40. Preceding OpId from leader: term: 1 index: 41. (index mismatch)
I20240501 14:09:07.396173 9014 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 41. (index mismatch)
I20240501 14:09:07.396641 9640 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 39, Time since last communication: 0.000s
I20240501 14:09:07.398676 9640 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20240501 14:09:07.399713 8749 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 40. Preceding OpId from leader: term: 1 index: 41. (index mismatch)
I20240501 14:09:07.400095 9640 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20240501 14:09:07.404227 10076 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 LEADER]: Committing config change with OpId 1.41: config changed from index -1 to 41, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.407016 9016 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.41: config changed from index -1 to 41, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.407428 9407 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.41: config changed from index -1 to 41, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.411410 8882 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 40. Preceding OpId from leader: term: 1 index: 41. (index mismatch)
I20240501 14:09:07.411902 10509 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20240501 14:09:07.420017 8882 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.41: config changed from index -1 to 41, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.426281 9013 raft_consensus.cc:1203] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.39->[1.40-1.40] Dedup: 1.40->[]
I20240501 14:09:07.437835 8749 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.41: config changed from index -1 to 41, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.469470 9406 consensus_queue.cc:237] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 47, Committed index: 47, Last appended: 1.47, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:07.471377 8885 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 47. Preceding OpId from leader: term: 1 index: 48. (index mismatch)
I20240501 14:09:07.472031 9963 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 47, Time since last communication: 0.000s
I20240501 14:09:07.474433 9144 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 47. Preceding OpId from leader: term: 1 index: 48. (index mismatch)
I20240501 14:09:07.475059 10482 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 47, Time since last communication: 0.000s
I20240501 14:09:07.476845 10482 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 LEADER]: Committing config change with OpId 1.48: config changed from index -1 to 48, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.479667 8883 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.48: config changed from index -1 to 48, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.479925 8752 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 47. Preceding OpId from leader: term: 1 index: 48. (index mismatch)
I20240501 14:09:07.480906 10482 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 47, Time since last communication: 0.000s
I20240501 14:09:07.482026 8752 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.48: config changed from index -1 to 48, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.486351 9147 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.48: config changed from index -1 to 48, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.489751 8612 catalog_manager.cc:5484] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index -1 to 48, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" committed_config { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.490044 8612 catalog_manager.cc:5484] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index -1 to 41, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" committed_config { opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.491636 9278 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 47. Preceding OpId from leader: term: 1 index: 48. (index mismatch)
I20240501 14:09:07.491988 10482 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 47, Time since last communication: 0.000s
I20240501 14:09:07.493199 9277 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.48: config changed from index -1 to 48, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
W20240501 14:09:07.501022 9343 consensus_peers.cc:485] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b2fe0f030204ecca740b14dfd6605ff. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:07.653527 10589 ts_tablet_manager.cc:902] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 25737230978048598f426b1a99b27441 (127.6.153.193:41195)
I20240501 14:09:07.708214 10589 tablet_copy_client.cc:308] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:41195
I20240501 14:09:07.717298 9016 consensus_queue.cc:237] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 55, Committed index: 55, Last appended: 1.55, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:07.723134 9146 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240501 14:09:07.723613 10542 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20240501 14:09:07.731321 9406 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240501 14:09:07.731763 10542 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20240501 14:09:07.732621 8774 tablet_copy_service.cc:140] P 25737230978048598f426b1a99b27441: Received BeginTabletCopySession request for tablet d21fe5dd65fe4fbea85fecc8ce19bc43 from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:48145)
I20240501 14:09:07.732779 8774 tablet_copy_service.cc:161] P 25737230978048598f426b1a99b27441: Beginning new tablet copy session on tablet d21fe5dd65fe4fbea85fecc8ce19bc43 from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:48145: session id = 9a65600939334f6db8dcca7f07923ece-d21fe5dd65fe4fbea85fecc8ce19bc43
I20240501 14:09:07.734256 8774 tablet_copy_source_session.cc:205] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:07.735705 10542 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.737581 8749 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240501 14:09:07.738262 9406 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.739348 8603 catalog_manager.cc:5484] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:07.740199 9142 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.741556 10103 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20240501 14:09:07.742692 8749 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.758162 9277 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240501 14:09:07.765084 10103 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20240501 14:09:07.773124 10589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21fe5dd65fe4fbea85fecc8ce19bc43. 1 dirs total, 0 dirs full, 0 dirs failed
W20240501 14:09:07.806607 8950 consensus_peers.cc:485] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e72446fcffed439c8653937330e5b8ee. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:07.810914 9277 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:07.879371 10608 ts_tablet_manager.cc:902] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983)
I20240501 14:09:07.884091 10608 tablet_copy_client.cc:308] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:34983
I20240501 14:09:07.891561 8752 consensus_queue.cc:237] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 60, Committed index: 60, Last appended: 1.61, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } }
I20240501 14:09:07.893580 9144 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240501 14:09:07.893935 10323 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 14:09:07.896644 9167 tablet_copy_service.cc:140] P 73ea0b7a5e4c45a3814d46e74d62331a: Received BeginTabletCopySession request for tablet 858bf448fd5249f0a86d17d6d011a6b4 from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:39303)
I20240501 14:09:07.896775 9167 tablet_copy_service.cc:161] P 73ea0b7a5e4c45a3814d46e74d62331a: Beginning new tablet copy session on tablet 858bf448fd5249f0a86d17d6d011a6b4 from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:39303: session id = 9a65600939334f6db8dcca7f07923ece-858bf448fd5249f0a86d17d6d011a6b4
I20240501 14:09:07.898103 9167 tablet_copy_source_session.cc:205] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:07.898285 8881 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240501 14:09:07.898890 10323 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 14:09:07.901211 9409 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240501 14:09:07.901180 10322 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.62: config changed from index -1 to 62, NON_VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) added. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } } }
I20240501 14:09:07.904191 8603 catalog_manager.cc:5484] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index -1 to 62, NON_VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) added. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:07.908666 9144 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index -1 to 62, NON_VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) added. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } } }
I20240501 14:09:07.909096 8882 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index -1 to 62, NON_VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) added. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } } }
I20240501 14:09:07.911077 10322 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 14:09:07.916324 9409 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index -1 to 62, NON_VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) added. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } } }
I20240501 14:09:07.918399 9277 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240501 14:09:07.926638 10608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858bf448fd5249f0a86d17d6d011a6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:07.929328 10608 tablet_copy_client.cc:783] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:07.929520 10608 tablet_copy_client.cc:647] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:07.937592 10411 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 60, Time since last communication: 0.000s
W20240501 14:09:07.938042 8690 consensus_peers.cc:485] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 -> Peer 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719): Couldn't send request to peer 84c7d91c82524b69a9184fcf1949b5b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 560c845cf74c4df1bf33732abc84620d. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:07.941848 10608 tablet_copy_client.cc:523] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:07.944993 10608 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:07.959061 9277 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index -1 to 62, NON_VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) added. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } } }
I20240501 14:09:07.960434 10589 tablet_copy_client.cc:783] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:07.961349 10589 tablet_copy_client.cc:647] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:07.962410 10614 ts_tablet_manager.cc:902] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:07.963033 10614 tablet_copy_client.cc:308] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:42941
I20240501 14:09:07.964239 10589 tablet_copy_client.cc:523] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:07.965893 10589 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:08.077611 10633 ts_tablet_manager.cc:902] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:08.080058 10633 tablet_copy_client.cc:308] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:32925
I20240501 14:09:08.111990 10589 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=60 overwritten=0 applied=60 ignored=0} inserts{seen=745 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:08.112349 10589 tablet_bootstrap.cc:492] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:08.112567 10589 ts_tablet_manager.cc:1367] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.147s user 0.063s sys 0.004s
I20240501 14:09:08.113229 10589 raft_consensus.cc:348] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.113374 10589 raft_consensus.cc:729] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:08.113500 10589 consensus_queue.cc:260] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 60, Last appended: 1.60, Last appended by leader: 60, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.113952 10589 ts_tablet_manager.cc:1398] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:08.116312 8774 tablet_copy_service.cc:342] P 25737230978048598f426b1a99b27441: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-d21fe5dd65fe4fbea85fecc8ce19bc43 received from {username='slave'} at 127.6.153.199:48145
I20240501 14:09:08.116425 8774 tablet_copy_service.cc:433] P 25737230978048598f426b1a99b27441: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-d21fe5dd65fe4fbea85fecc8ce19bc43 on tablet d21fe5dd65fe4fbea85fecc8ce19bc43 with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:08.121429 10244 raft_consensus.cc:1203] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.55->[1.56-1.62] Dedup: 1.60->[1.61-1.62]
W20240501 14:09:08.123871 9211 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.198:32925, user_credentials={real_user=slave}} blocked reactor thread for 52195.1us
I20240501 14:09:08.141990 9429 tablet_copy_service.cc:140] P 6b961a167ae64dbc9d102ce5ddf721f3: Received BeginTabletCopySession request for tablet 4b2fe0f030204ecca740b14dfd6605ff from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:41535)
I20240501 14:09:08.142174 9429 tablet_copy_service.cc:161] P 6b961a167ae64dbc9d102ce5ddf721f3: Beginning new tablet copy session on tablet 4b2fe0f030204ecca740b14dfd6605ff from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:41535: session id = 9a65600939334f6db8dcca7f07923ece-4b2fe0f030204ecca740b14dfd6605ff
I20240501 14:09:08.143760 9429 tablet_copy_source_session.cc:205] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:08.148208 10633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2fe0f030204ecca740b14dfd6605ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:08.150651 10633 tablet_copy_client.cc:783] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:08.150822 10633 tablet_copy_client.cc:647] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:08.155822 9298 tablet_copy_service.cc:140] P 0c0d3d3aae054876a0a522a8325ddd02: Received BeginTabletCopySession request for tablet 2d86cde53ceb4b63842ea9ec09659318 from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:47719)
I20240501 14:09:08.155978 9298 tablet_copy_service.cc:161] P 0c0d3d3aae054876a0a522a8325ddd02: Beginning new tablet copy session on tablet 2d86cde53ceb4b63842ea9ec09659318 from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:47719: session id = 9a65600939334f6db8dcca7f07923ece-2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:08.157310 9298 tablet_copy_source_session.cc:205] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:08.158443 10614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d86cde53ceb4b63842ea9ec09659318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:08.160671 10614 tablet_copy_client.cc:783] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:08.160825 10614 tablet_copy_client.cc:647] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:08.168300 10633 tablet_copy_client.cc:523] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:08.170411 10633 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:08.176134 10614 tablet_copy_client.cc:523] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:08.178843 10614 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:08.229595 10632 raft_consensus.cc:1050] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:08.230680 10632 consensus_queue.cc:237] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 62, Committed index: 62, Last appended: 1.63, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:08.232811 10244 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 14:09:08.233282 9142 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 14:09:08.233662 10488 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 14:09:08.233902 10411 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 14:09:08.240659 8885 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 14:09:08.244041 9409 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 14:09:08.247004 10488 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 14:09:08.250717 9014 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 14:09:08.254595 10488 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 14:09:08.256208 10488 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.64: config changed from index 56 to 64, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.258702 10244 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 56 to 64, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.260820 8885 raft_consensus.cc:1203] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.62->[1.63-1.64] Dedup: 1.63->[1.64-1.64]
I20240501 14:09:08.261914 8609 catalog_manager.cc:5484] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece reported cstate change: config changed from index 56 to 64, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.267719 9142 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 56 to 64, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.271093 9013 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 56 to 64, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.277603 10323 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 14:09:08.280040 9409 raft_consensus.cc:1203] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.62->[1.63-1.65] Dedup: 1.63->[1.64-1.65]
I20240501 14:09:08.280683 9409 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 56 to 64, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.285169 8883 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 56 to 64, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.326167 10589 ts_tablet_manager.cc:902] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719)
I20240501 14:09:08.326553 8752 consensus_queue.cc:237] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.66, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:08.326719 10589 tablet_copy_client.cc:308] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:33719
I20240501 14:09:08.378329 10244 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 14:09:08.378758 10411 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 14:09:08.380077 9144 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 14:09:08.381032 10632 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 64, Time since last communication: 0.000s
I20240501 14:09:08.386094 8883 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 14:09:08.386437 10322 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.386765 9407 raft_consensus.cc:1261] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 14:09:08.392830 9144 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.406368 8597 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d21fe5dd65fe4fbea85fecc8ce19bc43 with cas_config_opid_index 64: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:08.406893 10244 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.414670 8609 catalog_manager.cc:5484] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 64 to 67, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:08.422495 8996 tablet_service.cc:1514] Processing DeleteTablet for tablet d21fe5dd65fe4fbea85fecc8ce19bc43 with delete_type TABLET_DATA_TOMBSTONED (TS 84c7d91c82524b69a9184fcf1949b5b0 not found in new config with opid_index 67) from {username='slave'} at 127.0.0.1:34080
I20240501 14:09:08.426401 9851 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 14:09:08.426738 9851 consensus_queue.cc:1035] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 14:09:08.430440 9036 tablet_copy_service.cc:140] P 84c7d91c82524b69a9184fcf1949b5b0: Received BeginTabletCopySession request for tablet e72446fcffed439c8653937330e5b8ee from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:54759)
I20240501 14:09:08.430603 9036 tablet_copy_service.cc:161] P 84c7d91c82524b69a9184fcf1949b5b0: Beginning new tablet copy session on tablet e72446fcffed439c8653937330e5b8ee from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:54759: session id = 9a65600939334f6db8dcca7f07923ece-e72446fcffed439c8653937330e5b8ee
I20240501 14:09:08.433055 9036 tablet_copy_source_session.cc:205] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:08.440809 10679 tablet_replica.cc:330] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: stopping tablet replica
I20240501 14:09:08.441234 10679 raft_consensus.cc:2229] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:08.441561 10679 raft_consensus.cc:2258] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:08.453562 8882 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.455523 9409 raft_consensus.cc:2932] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:08.457006 10589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72446fcffed439c8653937330e5b8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:08.459313 10589 tablet_copy_client.cc:783] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:08.463207 10589 tablet_copy_client.cc:647] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:08.466985 10614 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=43 overwritten=0 applied=42 ignored=0} inserts{seen=673 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240501 14:09:08.473255 10614 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:08.473574 10614 ts_tablet_manager.cc:1367] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.295s user 0.053s sys 0.001s
I20240501 14:09:08.474151 10614 raft_consensus.cc:348] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.474862 10614 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:08.475051 10614 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 42, Last appended: 1.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.477375 10614 ts_tablet_manager.cc:1398] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:09:08.477510 10608 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=49 overwritten=0 applied=49 ignored=0} inserts{seen=734 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:08.479610 10608 tablet_bootstrap.cc:492] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:08.482059 10608 ts_tablet_manager.cc:1367] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.537s user 0.061s sys 0.005s
I20240501 14:09:08.483052 9298 tablet_copy_service.cc:342] P 0c0d3d3aae054876a0a522a8325ddd02: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-2d86cde53ceb4b63842ea9ec09659318 received from {username='slave'} at 127.6.153.199:47719
I20240501 14:09:08.483147 9298 tablet_copy_service.cc:433] P 0c0d3d3aae054876a0a522a8325ddd02: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-2d86cde53ceb4b63842ea9ec09659318 on tablet 2d86cde53ceb4b63842ea9ec09659318 with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:08.482764 10608 raft_consensus.cc:348] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.484367 10608 raft_consensus.cc:729] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:08.485836 10608 consensus_queue.cc:260] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 49, Last appended: 1.49, Last appended by leader: 49, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 41 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.488687 10608 ts_tablet_manager.cc:1398] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240501 14:09:08.491163 9167 tablet_copy_service.cc:342] P 73ea0b7a5e4c45a3814d46e74d62331a: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-858bf448fd5249f0a86d17d6d011a6b4 received from {username='slave'} at 127.6.153.199:39303
I20240501 14:09:08.491289 9167 tablet_copy_service.cc:433] P 73ea0b7a5e4c45a3814d46e74d62331a: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-858bf448fd5249f0a86d17d6d011a6b4 on tablet 858bf448fd5249f0a86d17d6d011a6b4 with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:08.506542 10589 tablet_copy_client.cc:523] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:08.532160 10589 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:08.546582 10682 ts_tablet_manager.cc:902] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: Initiating tablet copy from peer 25737230978048598f426b1a99b27441 (127.6.153.193:41195)
I20240501 14:09:08.547899 10682 tablet_copy_client.cc:308] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:41195
W20240501 14:09:08.570894 8950 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.193:41195, user_credentials={real_user=slave}} blocked reactor thread for 45120.6us
I20240501 14:09:08.592814 10679 ts_tablet_manager.cc:1875] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:08.604168 8774 tablet_copy_service.cc:140] P 25737230978048598f426b1a99b27441: Received BeginTabletCopySession request for tablet 560c845cf74c4df1bf33732abc84620d from peer 84c7d91c82524b69a9184fcf1949b5b0 ({username='slave'} at 127.6.153.195:37001)
I20240501 14:09:08.604312 8774 tablet_copy_service.cc:161] P 25737230978048598f426b1a99b27441: Beginning new tablet copy session on tablet 560c845cf74c4df1bf33732abc84620d from peer 84c7d91c82524b69a9184fcf1949b5b0 at {username='slave'} at 127.6.153.195:37001: session id = 84c7d91c82524b69a9184fcf1949b5b0-560c845cf74c4df1bf33732abc84620d
I20240501 14:09:08.605237 8774 tablet_copy_source_session.cc:205] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:08.640954 10682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 560c845cf74c4df1bf33732abc84620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:08.643780 10682 tablet_copy_client.cc:783] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:08.643987 10682 tablet_copy_client.cc:647] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:08.655237 10589 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=62 overwritten=0 applied=62 ignored=0} inserts{seen=734 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:08.655839 10589 tablet_bootstrap.cc:492] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:08.656059 10589 ts_tablet_manager.cc:1367] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.124s user 0.061s sys 0.004s
I20240501 14:09:08.656812 10589 raft_consensus.cc:348] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.656972 10589 raft_consensus.cc:729] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:08.657086 10589 consensus_queue.cc:260] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 62, Last appended: 1.62, Last appended by leader: 62, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.658193 10589 ts_tablet_manager.cc:1398] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:08.658754 9036 tablet_copy_service.cc:342] P 84c7d91c82524b69a9184fcf1949b5b0: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-e72446fcffed439c8653937330e5b8ee received from {username='slave'} at 127.6.153.199:54759
I20240501 14:09:08.658869 9036 tablet_copy_service.cc:433] P 84c7d91c82524b69a9184fcf1949b5b0: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-e72446fcffed439c8653937330e5b8ee on tablet e72446fcffed439c8653937330e5b8ee with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:08.660641 10679 ts_tablet_manager.cc:1888] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.65
I20240501 14:09:08.660799 10679 log.cc:1196] T d21fe5dd65fe4fbea85fecc8ce19bc43 P 84c7d91c82524b69a9184fcf1949b5b0: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/d21fe5dd65fe4fbea85fecc8ce19bc43
I20240501 14:09:08.661512 8599 catalog_manager.cc:4830] TS 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719): tablet d21fe5dd65fe4fbea85fecc8ce19bc43 (table rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]) successfully deleted
I20240501 14:09:08.678786 10682 tablet_copy_client.cc:523] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:08.682592 10682 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap starting.
I20240501 14:09:08.725422 10633 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=59 overwritten=0 applied=59 ignored=0} inserts{seen=690 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:08.725726 10633 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:08.725894 10633 ts_tablet_manager.cc:1367] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.556s user 0.056s sys 0.007s
I20240501 14:09:08.726310 10633 raft_consensus.cc:348] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.729790 10633 raft_consensus.cc:729] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:08.729943 10633 consensus_queue.cc:260] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 59, Last appended: 1.59, Last appended by leader: 59, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 48 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:08.730345 10633 ts_tablet_manager.cc:1398] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20240501 14:09:08.742822 9429 tablet_copy_service.cc:342] P 6b961a167ae64dbc9d102ce5ddf721f3: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-4b2fe0f030204ecca740b14dfd6605ff received from {username='slave'} at 127.6.153.199:41535
I20240501 14:09:08.742950 9429 tablet_copy_service.cc:433] P 6b961a167ae64dbc9d102ce5ddf721f3: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-4b2fe0f030204ecca740b14dfd6605ff on tablet 4b2fe0f030204ecca740b14dfd6605ff with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:08.883562 9014 consensus_queue.cc:237] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 77, Committed index: 77, Last appended: 1.77, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
W20240501 14:09:08.886480 8950 consensus_peers.cc:485] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4323f538490b4bf99fa7f96f80c8a88f. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:08.888481 8750 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 14:09:08.889251 10103 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 14:09:08.894997 8880 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 14:09:08.896330 10695 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 76, Time since last communication: 0.000s
I20240501 14:09:08.904875 10244 raft_consensus.cc:1203] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.55->[1.56-1.67] Dedup: 1.62->[1.63-1.67]
I20240501 14:09:08.907088 10243 raft_consensus.cc:1203] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.41->[1.42-1.53] Dedup: 1.43->[1.44-1.53]
I20240501 14:09:08.911788 9146 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 14:09:08.913075 10542 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 14:09:08.915576 9278 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 76. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 14:09:08.922669 10423 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.78: config changed from index -1 to 78, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:08.924548 9147 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index -1 to 78, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:08.925977 8603 catalog_manager.cc:5484] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index -1 to 78, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:08.931263 8753 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index -1 to 78, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:08.935632 10695 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 75, Time since last communication: 0.000s
I20240501 14:09:08.953274 9274 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index -1 to 78, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:08.983083 8881 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index -1 to 78, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:09.006021 10672 raft_consensus.cc:1050] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:09.006559 10672 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 53, Committed index: 53, Last appended: 1.54, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.008042 9405 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 14:09:09.008155 9146 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 53. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 14:09:09.008220 10243 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 14:09:09.008432 10344 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 14:09:09.008551 9948 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 14:09:09.010541 10565 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 14:09:09.014086 10367 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.55: config changed from index 42 to 55, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.016130 9147 raft_consensus.cc:1203] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.53->[1.54-1.54] Dedup: 1.54->[]
I20240501 14:09:09.016657 10243 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index 42 to 55, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.019515 9146 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index 42 to 55, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.019894 8607 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece reported cstate change: config changed from index 42 to 55, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.021152 9405 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index 42 to 55, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.022042 10244 raft_consensus.cc:1203] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.40->[1.41-1.72] Dedup: 1.49->[1.50-1.72]
I20240501 14:09:09.028851 9014 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 53. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 14:09:09.029618 8883 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 14:09:09.033659 10367 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 14:09:09.036543 10542 raft_consensus.cc:1050] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:09.039543 10542 consensus_queue.cc:237] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 68, Committed index: 68, Last appended: 1.68, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.041790 9408 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 14:09:09.042217 10695 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 68, Time since last communication: 0.000s
I20240501 14:09:09.043614 10242 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 14:09:09.043956 9147 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 14:09:09.044626 10695 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 68, Time since last communication: 0.000s
I20240501 14:09:09.046130 10695 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 68, Time since last communication: 0.000s
I20240501 14:09:09.049578 10542 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.69: config changed from index 56 to 69, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.051645 9405 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 56 to 69, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.053094 9144 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 56 to 69, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.055785 10242 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 56 to 69, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.071151 10509 raft_consensus.cc:1050] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:09.071710 10509 consensus_queue.cc:237] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 72, Committed index: 72, Last appended: 1.73, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.073076 10244 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.073550 10140 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 14:09:09.074493 9014 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.074595 9407 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 72. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.075043 10076 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 14:09:09.077555 8753 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 14:09:09.077899 10076 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 14:09:09.079425 9407 raft_consensus.cc:1203] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.72->[1.73-1.74] Dedup: 1.73->[1.74-1.74]
I20240501 14:09:09.081094 10140 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 LEADER]: Committing config change with OpId 1.74: config changed from index 41 to 74, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.082358 10682 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=74 overwritten=0 applied=74 ignored=0} inserts{seen=759 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:09.082667 10682 tablet_bootstrap.cc:492] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: Bootstrap complete.
I20240501 14:09:09.082927 10682 ts_tablet_manager.cc:1367] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: Time spent bootstrapping tablet: real 0.400s user 0.070s sys 0.008s
I20240501 14:09:09.082886 8881 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index 42 to 55, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.083442 10682 raft_consensus.cc:348] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } }
I20240501 14:09:09.083552 10682 raft_consensus.cc:729] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Initialized, Role: LEARNER
I20240501 14:09:09.083636 10682 consensus_queue.cc:260] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 74, Last appended: 1.74, Last appended by leader: 74, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: true } }
I20240501 14:09:09.084007 10682 ts_tablet_manager.cc:1398] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:09.084582 10244 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 41 to 74, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.084720 9014 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 41 to 74, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.087092 9407 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 41 to 74, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.096117 8885 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 67. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.103826 8774 tablet_copy_service.cc:342] P 25737230978048598f426b1a99b27441: Request end of tablet copy session 84c7d91c82524b69a9184fcf1949b5b0-560c845cf74c4df1bf33732abc84620d received from {username='slave'} at 127.6.153.195:37001
I20240501 14:09:09.103946 8774 tablet_copy_service.cc:433] P 25737230978048598f426b1a99b27441: ending tablet copy session 84c7d91c82524b69a9184fcf1949b5b0-560c845cf74c4df1bf33732abc84620d on tablet 560c845cf74c4df1bf33732abc84620d with peer 84c7d91c82524b69a9184fcf1949b5b0
I20240501 14:09:09.105221 8753 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.105513 10542 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240501 14:09:09.106487 10367 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 14:09:09.109932 10509 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 67, Time since last communication: 0.000s
I20240501 14:09:09.127874 9277 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 14:09:09.132155 9014 raft_consensus.cc:1203] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.53->[1.54-1.55] Dedup: 1.54->[1.55-1.55]
I20240501 14:09:09.133031 9014 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index 42 to 55, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.135879 10696 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240501 14:09:09.139081 10509 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 14:09:09.149520 8884 raft_consensus.cc:1203] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.67->[1.68-1.74] Dedup: 1.72->[1.73-1.74]
I20240501 14:09:09.157188 8884 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 41 to 74, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.162281 10244 raft_consensus.cc:1203] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.47->[1.48-1.70] Dedup: 1.59->[1.60-1.70]
I20240501 14:09:09.175248 9276 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 56 to 69, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.182008 8751 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 41 to 74, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.182190 8753 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 56 to 69, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.263706 8609 catalog_manager.cc:5484] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 56 to 69, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.264046 8609 catalog_manager.cc:5484] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 41 to 74, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" committed_config { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.275115 9013 consensus_queue.cc:237] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 77, Committed index: 77, Last appended: 1.78, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.278609 9405 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 14:09:09.279388 10414 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 14:09:09.279537 10244 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 14:09:09.279753 9146 consensus_queue.cc:237] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 75, Committed index: 75, Last appended: 1.75, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 76 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.280984 10414 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 14:09:09.281494 9014 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 75. Preceding OpId from leader: term: 1 index: 76. (index mismatch)
I20240501 14:09:09.281855 10582 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 76, Last known committed idx: 75, Time since last communication: 0.000s
I20240501 14:09:09.281920 10244 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 75. Preceding OpId from leader: term: 1 index: 76. (index mismatch)
I20240501 14:09:09.282382 9144 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 14:09:09.282406 10582 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 76, Last known committed idx: 75, Time since last communication: 0.000s
I20240501 14:09:09.283098 10498 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 14:09:09.283607 10498 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.79: config changed from index 69 to 79, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.286489 8607 catalog_manager.cc:5484] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index 69 to 79, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:09.287695 10242 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index 69 to 79, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.288058 8599 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e72446fcffed439c8653937330e5b8ee with cas_config_opid_index 69: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:09.288350 10582 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 LEADER]: Committing config change with OpId 1.76: config changed from index 74 to 76, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 76 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.289829 8598 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 858bf448fd5249f0a86d17d6d011a6b4 with cas_config_opid_index 74: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:09.290531 9407 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index 69 to 79, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.291121 8612 catalog_manager.cc:5484] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a reported cstate change: config changed from index 74 to 76, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New cstate: current_term: 1 leader_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" committed_config { opid_index: 76 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:09.291115 9014 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.76: config changed from index 74 to 76, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 76 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.293140 10242 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.76: config changed from index 74 to 76, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 76 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.293824 9144 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index 69 to 79, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.295166 8732 tablet_service.cc:1514] Processing DeleteTablet for tablet e72446fcffed439c8653937330e5b8ee with delete_type TABLET_DATA_TOMBSTONED (TS 25737230978048598f426b1a99b27441 not found in new config with opid_index 79) from {username='slave'} at 127.0.0.1:57504
I20240501 14:09:09.296289 10786 tablet_replica.cc:330] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: stopping tablet replica
I20240501 14:09:09.296878 10786 raft_consensus.cc:2229] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:09.297238 10786 raft_consensus.cc:2258] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:09.298875 9383 tablet_service.cc:1514] Processing DeleteTablet for tablet 858bf448fd5249f0a86d17d6d011a6b4 with delete_type TABLET_DATA_TOMBSTONED (TS 6b961a167ae64dbc9d102ce5ddf721f3 not found in new config with opid_index 76) from {username='slave'} at 127.0.0.1:39116
I20240501 14:09:09.299400 10787 tablet_replica.cc:330] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: stopping tablet replica
I20240501 14:09:09.299716 10787 raft_consensus.cc:2229] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:09.299798 10786 ts_tablet_manager.cc:1875] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:09.300007 10787 raft_consensus.cc:2258] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:09.301187 10787 ts_tablet_manager.cc:1875] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:09.303834 10787 ts_tablet_manager.cc:1888] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.75
I20240501 14:09:09.303982 10787 log.cc:1196] T 858bf448fd5249f0a86d17d6d011a6b4 P 6b961a167ae64dbc9d102ce5ddf721f3: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/wals/858bf448fd5249f0a86d17d6d011a6b4
I20240501 14:09:09.305049 10786 ts_tablet_manager.cc:1888] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.76
I20240501 14:09:09.305328 10786 log.cc:1196] T e72446fcffed439c8653937330e5b8ee P 25737230978048598f426b1a99b27441: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/e72446fcffed439c8653937330e5b8ee
I20240501 14:09:09.306567 8597 catalog_manager.cc:4830] TS 25737230978048598f426b1a99b27441 (127.6.153.193:41195): tablet e72446fcffed439c8653937330e5b8ee (table rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]) successfully deleted
I20240501 14:09:09.306794 8596 catalog_manager.cc:4830] TS 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925): tablet 858bf448fd5249f0a86d17d6d011a6b4 (table rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]) successfully deleted
I20240501 14:09:09.311264 9014 raft_consensus.cc:1203] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.61->[1.62-1.83] Dedup: 1.74->[1.75-1.83]
I20240501 14:09:09.335479 9277 raft_consensus.cc:1261] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 14:09:09.346410 10322 raft_consensus.cc:1050] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441: attempting to promote NON_VOTER 84c7d91c82524b69a9184fcf1949b5b0 to VOTER
I20240501 14:09:09.346616 8751 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 75. Preceding OpId from leader: term: 1 index: 76. (index mismatch)
I20240501 14:09:09.346736 10414 consensus_queue.cc:1035] T e72446fcffed439c8653937330e5b8ee P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 76, Time since last communication: 0.000s
I20240501 14:09:09.346823 10322 consensus_queue.cc:237] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 84, Committed index: 84, Last appended: 1.84, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } }
I20240501 14:09:09.350864 9144 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240501 14:09:09.351308 10411 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 84, Time since last communication: 0.000s
I20240501 14:09:09.352044 9277 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240501 14:09:09.352550 9015 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEARNER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240501 14:09:09.353171 10411 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 84, Time since last communication: 0.000s
I20240501 14:09:09.353787 9851 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 83, Time since last communication: 0.000s
I20240501 14:09:09.354727 8881 raft_consensus.cc:1261] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 75. Preceding OpId from leader: term: 1 index: 76. (index mismatch)
I20240501 14:09:09.364596 10411 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.85: config changed from index 62 to 85, 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.364915 9013 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 62 to 85, 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.367517 9147 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 62 to 85, 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.367722 9277 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 62 to 85, 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.371300 8608 catalog_manager.cc:5484] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a reported cstate change: config changed from index 62 to 85, 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.374466 9640 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 76, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 14:09:09.386554 9640 consensus_queue.cc:1035] T 858bf448fd5249f0a86d17d6d011a6b4 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 76, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 14:09:09.387008 10583 raft_consensus.cc:1050] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:09.387554 10583 consensus_queue.cc:237] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 70, Committed index: 70, Last appended: 1.71, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.389011 10244 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 14:09:09.389230 8881 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 14:09:09.389518 9277 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 14:09:09.389657 9963 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 70, Time since last communication: 0.000s
I20240501 14:09:09.389935 9963 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 70, Time since last communication: 0.000s
I20240501 14:09:09.390173 9963 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 70, Time since last communication: 0.000s
I20240501 14:09:09.390942 9146 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 14:09:09.392570 9277 raft_consensus.cc:2932] T e72446fcffed439c8653937330e5b8ee P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index 69 to 79, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.394791 10737 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 70, Time since last communication: 0.000s
I20240501 14:09:09.396540 10737 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 LEADER]: Committing config change with OpId 1.72: config changed from index 48 to 72, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.397601 9144 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 48 to 72, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.398545 9276 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 48 to 72, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.398763 8881 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 48 to 72, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.400779 10244 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 48 to 72, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.401320 8750 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 14:09:09.401369 8607 catalog_manager.cc:5484] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a reported cstate change: config changed from index 48 to 72, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" committed_config { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.403486 10482 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 70, Time since last communication: 0.000s
I20240501 14:09:09.406028 9276 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 58, Committed index: 58, Last appended: 1.58, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.406919 8750 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 48 to 72, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.407595 9408 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 14:09:09.408048 10565 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20240501 14:09:09.409621 10242 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 14:09:09.410421 9144 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 14:09:09.410758 10565 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20240501 14:09:09.410890 10650 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20240501 14:09:09.411118 8881 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240501 14:09:09.411890 9408 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240501 14:09:09.413479 10565 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.59: config changed from index 55 to 59, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.415014 9144 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 55 to 59, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.415617 10242 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 55 to 59, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.417415 8606 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a reported cstate change: config changed from index 55 to 59, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.418390 9408 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 55 to 59, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.419484 9013 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 14:09:09.421006 8863 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d86cde53ceb4b63842ea9ec09659318 with delete_type TABLET_DATA_TOMBSTONED (TS f2e6587c4d054aa696aaa7ff76880a2f not found in new config with opid_index 59) from {username='slave'} at 127.0.0.1:57374
I20240501 14:09:09.421943 8754 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.76: config changed from index 74 to 76, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 76 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.425546 10632 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 83, Time since last communication: 0.000s
I20240501 14:09:09.431469 8598 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2d86cde53ceb4b63842ea9ec09659318 with cas_config_opid_index 55: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:09.439244 10322 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 83, Time since last communication: 0.000s
I20240501 14:09:09.442098 8881 raft_consensus.cc:2932] T 858bf448fd5249f0a86d17d6d011a6b4 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.76: config changed from index 74 to 76, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 76 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.442322 10794 tablet_replica.cc:330] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: stopping tablet replica
I20240501 14:09:09.442637 10794 raft_consensus.cc:2229] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:09.442842 10794 raft_consensus.cc:2258] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:09.449396 10794 ts_tablet_manager.cc:1875] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:09.451434 10565 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20240501 14:09:09.452210 9408 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 62 to 85, 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.457039 8883 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 62 to 85, 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.460446 10794 ts_tablet_manager.cc:1888] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.58
I20240501 14:09:09.460563 10794 log.cc:1196] T 2d86cde53ceb4b63842ea9ec09659318 P f2e6587c4d054aa696aaa7ff76880a2f: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:09.461181 8596 catalog_manager.cc:4830] TS f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221): tablet 2d86cde53ceb4b63842ea9ec09659318 (table rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]) successfully deleted
I20240501 14:09:09.467233 9015 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 55 to 59, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.491415 9405 consensus_queue.cc:237] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 73, Committed index: 73, Last appended: 1.73, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.493940 10242 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.493990 8750 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.494297 10737 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 14:09:09.494817 9147 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.495195 10382 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 14:09:09.495481 10795 ts_tablet_manager.cc:902] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719)
I20240501 14:09:09.495673 8880 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:09.496068 10382 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 14:09:09.497665 10795 tablet_copy_client.cc:308] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:33719
I20240501 14:09:09.497545 10583 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 LEADER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.498450 8880 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.499981 8751 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.499995 10242 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.501282 10382 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 14:09:09.502843 9036 tablet_copy_service.cc:140] P 84c7d91c82524b69a9184fcf1949b5b0: Received BeginTabletCopySession request for tablet 4323f538490b4bf99fa7f96f80c8a88f from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:54759)
I20240501 14:09:09.502949 9036 tablet_copy_service.cc:161] P 84c7d91c82524b69a9184fcf1949b5b0: Beginning new tablet copy session on tablet 4323f538490b4bf99fa7f96f80c8a88f from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:54759: session id = 9a65600939334f6db8dcca7f07923ece-4323f538490b4bf99fa7f96f80c8a88f
I20240501 14:09:09.502907 8612 catalog_manager.cc:5484] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: config changed from index 72 to 74, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New cstate: current_term: 1 leader_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" committed_config { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.503921 9036 tablet_copy_source_session.cc:205] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:09.507125 10795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4323f538490b4bf99fa7f96f80c8a88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:09.507577 8596 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4b2fe0f030204ecca740b14dfd6605ff with cas_config_opid_index 72: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:09.510545 9144 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:09.523196 10795 tablet_copy_client.cc:783] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:09.523676 10795 tablet_copy_client.cc:647] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:09.525384 10795 tablet_copy_client.cc:523] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:09.526933 10795 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:09.534849 9248 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2fe0f030204ecca740b14dfd6605ff with delete_type TABLET_DATA_TOMBSTONED (TS 0c0d3d3aae054876a0a522a8325ddd02 not found in new config with opid_index 74) from {username='slave'} at 127.0.0.1:54040
W20240501 14:09:09.657584 8687 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.194:35221, user_credentials={real_user=slave}} blocked reactor thread for 101969us
I20240501 14:09:09.746980 10795 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=87 overwritten=0 applied=86 ignored=0} inserts{seen=771 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240501 14:09:09.747304 10795 tablet_bootstrap.cc:492] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:09.747536 10795 ts_tablet_manager.cc:1367] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.221s user 0.081s sys 0.006s
I20240501 14:09:09.747987 10795 raft_consensus.cc:348] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:09.751452 10795 raft_consensus.cc:729] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:09.748368 10806 tablet_replica.cc:330] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: stopping tablet replica
I20240501 14:09:09.751603 10795 consensus_queue.cc:260] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 86, Last appended: 1.87, Last appended by leader: 87, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:09.751943 10795 ts_tablet_manager.cc:1398] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:09:09.752537 9036 tablet_copy_service.cc:342] P 84c7d91c82524b69a9184fcf1949b5b0: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-4323f538490b4bf99fa7f96f80c8a88f received from {username='slave'} at 127.6.153.199:54759
I20240501 14:09:09.752624 9036 tablet_copy_service.cc:433] P 84c7d91c82524b69a9184fcf1949b5b0: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-4323f538490b4bf99fa7f96f80c8a88f on tablet 4323f538490b4bf99fa7f96f80c8a88f with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:09.762048 10806 raft_consensus.cc:2229] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:09.775522 10806 raft_consensus.cc:2258] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:09.794778 10806 ts_tablet_manager.cc:1875] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20240501 14:09:09.796140 8951 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.197:42941, user_credentials={real_user=slave}} blocked reactor thread for 46961.7us
I20240501 14:09:09.817688 10806 ts_tablet_manager.cc:1888] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.73
I20240501 14:09:09.818312 10806 log.cc:1196] T 4b2fe0f030204ecca740b14dfd6605ff P 0c0d3d3aae054876a0a522a8325ddd02: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/4b2fe0f030204ecca740b14dfd6605ff
I20240501 14:09:09.820225 8598 catalog_manager.cc:4830] TS 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941): tablet 4b2fe0f030204ecca740b14dfd6605ff (table rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]) successfully deleted
I20240501 14:09:09.847920 8751 consensus_queue.cc:237] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 85, Committed index: 85, Last appended: 1.86, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } }
I20240501 14:09:09.852967 9013 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 86. Preceding OpId from leader: term: 1 index: 87. (index mismatch)
I20240501 14:09:09.855909 10322 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 87, Last known committed idx: 85, Time since last communication: 0.000s
I20240501 14:09:09.860671 8881 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 86. Preceding OpId from leader: term: 1 index: 87. (index mismatch)
I20240501 14:09:09.861486 9277 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 86. Preceding OpId from leader: term: 1 index: 87. (index mismatch)
I20240501 14:09:09.862303 9408 raft_consensus.cc:1261] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 86. Preceding OpId from leader: term: 1 index: 87. (index mismatch)
I20240501 14:09:09.863924 10323 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 87, Last known committed idx: 85, Time since last communication: 0.000s
I20240501 14:09:09.864320 10323 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 87, Last known committed idx: 85, Time since last communication: 0.000s
I20240501 14:09:09.870595 10632 consensus_queue.cc:1035] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 87, Last known committed idx: 85, Time since last communication: 0.000s
I20240501 14:09:09.875545 10411 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.878007 9013 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.878576 9277 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.878851 8597 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 560c845cf74c4df1bf33732abc84620d with cas_config_opid_index 85: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:09.880115 8612 catalog_manager.cc:5484] T 560c845cf74c4df1bf33732abc84620d P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 85 to 87, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:09.882222 8881 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.886920 9408 raft_consensus.cc:2932] T 560c845cf74c4df1bf33732abc84620d P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { promote: false } } }
I20240501 14:09:09.892184 10242 raft_consensus.cc:1203] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.77->[1.78-1.94] Dedup: 1.87->[1.88-1.94]
I20240501 14:09:09.898491 9126 tablet_service.cc:1514] Processing DeleteTablet for tablet 560c845cf74c4df1bf33732abc84620d with delete_type TABLET_DATA_TOMBSTONED (TS 73ea0b7a5e4c45a3814d46e74d62331a not found in new config with opid_index 87) from {username='slave'} at 127.0.0.1:49360
I20240501 14:09:09.906298 10831 tablet_replica.cc:330] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: stopping tablet replica
I20240501 14:09:09.906651 10831 raft_consensus.cc:2229] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:09.906805 10831 pending_rounds.cc:62] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: Trying to abort 1 pending ops.
I20240501 14:09:09.906859 10831 pending_rounds.cc:68] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: Aborting op as it isn't in flight: id { term: 1 index: 86 } timestamp: 7022889164158148608 op_type: WRITE_OP write_request { tablet_id: "560c845cf74c4df1bf33732abc84620d" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\343\343\323\010\2019\006\002\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""f97ec37be2a7fd1b" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022889164038434816 authz_token { token_data: "\010\254\234\311\261\006\"3\n\005slave\022*\n 218e82ec93474fe58c6f5feec4b863de\020\001\030\001 \001(\001" signature: "<redacted>""\243\360\003Ua\016\376\303+\002\235\365\337\016\264\332l.v\276\212~7u\004p\347\244\360\227\375 \374G\327_\254\355}Qt\245\325W\000\017\254\002\324\212\277\207\rX\214\006\377\311!U\267\205J\"" signing_key_seq_num: 0 } } request_id { client_id: "03245ec4e6e84df184a1fb961b97bab1" seq_no: 212 first_incomplete_seq_no: 212 attempt_no: 0 }
I20240501 14:09:09.907194 10831 raft_consensus.cc:2258] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:09.911453 10831 ts_tablet_manager.cc:1875] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:09.921659 10831 ts_tablet_manager.cc:1888] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.86
I20240501 14:09:09.921782 10831 log.cc:1196] T 560c845cf74c4df1bf33732abc84620d P 73ea0b7a5e4c45a3814d46e74d62331a: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/560c845cf74c4df1bf33732abc84620d
I20240501 14:09:09.922372 8598 catalog_manager.cc:4830] TS 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983): tablet 560c845cf74c4df1bf33732abc84620d (table rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]) successfully deleted
I20240501 14:09:09.949512 9902 raft_consensus.cc:1050] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:09.950002 9902 consensus_queue.cc:237] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 94, Committed index: 94, Last appended: 1.95, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:09.951687 10244 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 14:09:09.951738 8751 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 14:09:09.952025 10695 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 14:09:09.954974 10414 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 14:09:09.967725 9147 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 14:09:09.974210 9276 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 14:09:09.988685 10414 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 14:09:10.015655 8885 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 14:09:10.016796 10695 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 14:09:10.049180 10103 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.96: config changed from index 78 to 96, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.050758 10103 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 14:09:10.052946 8606 catalog_manager.cc:5484] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index 78 to 96, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:10.055465 10243 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 78 to 96, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.062182 8750 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 78 to 96, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
W20240501 14:09:10.108893 8951 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.197:42941, user_credentials={real_user=slave}} blocked reactor thread for 60170.4us
I20240501 14:09:10.115877 9276 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 78 to 96, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.127983 9147 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 78 to 96, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.141453 9014 consensus_queue.cc:237] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 96, Committed index: 96, Last appended: 1.97, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:10.143635 10242 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240501 14:09:10.144165 10414 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 96, Time since last communication: 0.000s
I20240501 14:09:10.145186 8885 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 78 to 96, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.148953 8750 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240501 14:09:10.149652 10414 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 96, Time since last communication: 0.000s
I20240501 14:09:10.155462 10695 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.98: config changed from index 96 to 98, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.159021 8606 catalog_manager.cc:5484] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index 96 to 98, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:10.161029 10242 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 96 to 98, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.161224 8750 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 96 to 98, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.169242 8599 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4323f538490b4bf99fa7f96f80c8a88f with cas_config_opid_index 96: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:10.169379 9144 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240501 14:09:10.173607 9276 raft_consensus.cc:1261] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 96. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240501 14:09:10.189914 8863 tablet_service.cc:1514] Processing DeleteTablet for tablet 4323f538490b4bf99fa7f96f80c8a88f with delete_type TABLET_DATA_TOMBSTONED (TS f2e6587c4d054aa696aaa7ff76880a2f not found in new config with opid_index 98) from {username='slave'} at 127.0.0.1:57374
I20240501 14:09:10.214913 10542 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 96, Time since last communication: 0.000s
I20240501 14:09:10.215427 10542 consensus_queue.cc:1035] T 4323f538490b4bf99fa7f96f80c8a88f P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 96, Time since last communication: 0.000s
I20240501 14:09:10.238421 10844 tablet_replica.cc:330] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: stopping tablet replica
I20240501 14:09:10.238654 10844 raft_consensus.cc:2229] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:10.238860 10844 raft_consensus.cc:2258] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:10.239940 10844 ts_tablet_manager.cc:1875] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:10.255617 10844 ts_tablet_manager.cc:1888] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.96
I20240501 14:09:10.255821 10844 log.cc:1196] T 4323f538490b4bf99fa7f96f80c8a88f P f2e6587c4d054aa696aaa7ff76880a2f: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/4323f538490b4bf99fa7f96f80c8a88f
I20240501 14:09:10.256394 8596 catalog_manager.cc:4830] TS f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221): tablet 4323f538490b4bf99fa7f96f80c8a88f (table rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]) successfully deleted
I20240501 14:09:10.320180 9147 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 96 to 98, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.321015 9277 raft_consensus.cc:2932] T 4323f538490b4bf99fa7f96f80c8a88f P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 96 to 98, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:10.473203 8881 consensus_queue.cc:237] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 113, Committed index: 113, Last appended: 1.113, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:10.475759 8750 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 113. Preceding OpId from leader: term: 1 index: 114. (index mismatch)
I20240501 14:09:10.476498 9406 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 113. Preceding OpId from leader: term: 1 index: 114. (index mismatch)
I20240501 14:09:10.476589 10808 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 114, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:10.477738 10808 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 114, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:10.478690 9146 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 113. Preceding OpId from leader: term: 1 index: 114. (index mismatch)
I20240501 14:09:10.479071 10808 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 114, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:10.479878 10808 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 LEADER]: Committing config change with OpId 1.114: config changed from index -1 to 114, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.483362 8612 catalog_manager.cc:5484] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: config changed from index -1 to 114, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:10.483862 8753 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.114: config changed from index -1 to 114, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.484027 9407 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.114: config changed from index -1 to 114, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.487870 9144 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.114: config changed from index -1 to 114, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.516927 9274 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 113. Preceding OpId from leader: term: 1 index: 114. (index mismatch)
I20240501 14:09:10.520576 10841 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 114, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:10.521876 9276 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.114: config changed from index -1 to 114, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
W20240501 14:09:10.538745 8819 consensus_peers.cc:485] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96ab0c0bda54481b95b0fdf9d8da84df. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:10.760023 9277 consensus_queue.cc:237] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 73, Committed index: 73, Last appended: 1.73, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
W20240501 14:09:10.762955 9212 consensus_peers.cc:485] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7eac39a2ecdc40f0929ab332508189cd. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:10.768692 9147 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:10.769665 10851 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 14:09:10.786851 9016 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:10.798908 9408 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:10.804426 10851 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 14:09:10.815423 8885 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 14:09:10.818475 10650 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 14:09:10.822777 10344 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 14:09:10.825073 10650 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.828572 9142 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.831826 8608 catalog_manager.cc:5484] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a reported cstate change: config changed from index -1 to 74, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.853888 9274 consensus_queue.cc:237] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 83, Committed index: 83, Last appended: 1.83, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
W20240501 14:09:10.855868 9212 consensus_peers.cc:485] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b39b2a74a9c4dd78fc5541b3ce52772. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:10.863835 9016 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.866258 9016 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 83. Preceding OpId from leader: term: 1 index: 84. (index mismatch)
I20240501 14:09:10.870637 10851 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 84, Last known committed idx: 83, Time since last communication: 0.000s
I20240501 14:09:10.877655 8885 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 83. Preceding OpId from leader: term: 1 index: 84. (index mismatch)
I20240501 14:09:10.881157 9405 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 83. Preceding OpId from leader: term: 1 index: 84. (index mismatch)
I20240501 14:09:10.885689 8753 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 83. Preceding OpId from leader: term: 1 index: 84. (index mismatch)
I20240501 14:09:10.888581 10674 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 84, Last known committed idx: 83, Time since last communication: 0.000s
I20240501 14:09:10.924152 10674 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.84: config changed from index -1 to 84, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.928735 8754 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.84: config changed from index -1 to 84, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.930335 10851 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 84, Last known committed idx: 83, Time since last communication: 0.000s
I20240501 14:09:10.930677 10851 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 84, Last known committed idx: 83, Time since last communication: 0.000s
I20240501 14:09:10.931509 8878 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.933120 9408 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.938570 8606 catalog_manager.cc:5484] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index -1 to 84, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.939513 9408 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.84: config changed from index -1 to 84, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.944034 9015 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.84: config changed from index -1 to 84, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:10.966344 8878 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.84: config changed from index -1 to 84, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
W20240501 14:09:11.025146 8949 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.198:32925, user_credentials={real_user=slave}} blocked reactor thread for 47756.1us
I20240501 14:09:11.060663 8754 consensus_queue.cc:237] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 98, Committed index: 98, Last appended: 1.98, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:11.065418 8885 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 98. Preceding OpId from leader: term: 1 index: 99. (index mismatch)
I20240501 14:09:11.066200 10845 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 99, Last known committed idx: 97, Time since last communication: 0.000s
I20240501 14:09:11.067216 9146 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 98. Preceding OpId from leader: term: 1 index: 99. (index mismatch)
I20240501 14:09:11.067692 10632 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 99, Last known committed idx: 98, Time since last communication: 0.000s
I20240501 14:09:11.069862 10845 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.99: config changed from index -1 to 99, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:11.070293 9146 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.99: config changed from index -1 to 99, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:11.074975 8608 catalog_manager.cc:5484] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index -1 to 99, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:11.076262 8885 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.99: config changed from index -1 to 99, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:11.079936 9013 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 98. Preceding OpId from leader: term: 1 index: 99. (index mismatch)
I20240501 14:09:11.083609 10845 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 99, Last known committed idx: 98, Time since last communication: 0.000s
W20240501 14:09:11.086481 8688 consensus_peers.cc:485] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 -> Peer 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): Couldn't send request to peer 9a65600939334f6db8dcca7f07923ece. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6b7f72cb79d480a8a39e20c51d922f9. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:11.089520 9276 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 98. Preceding OpId from leader: term: 1 index: 99. (index mismatch)
I20240501 14:09:11.091491 10845 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 99, Last known committed idx: 98, Time since last communication: 0.000s
I20240501 14:09:11.093098 9277 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.99: config changed from index -1 to 99, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:11.098064 9014 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.99: config changed from index -1 to 99, NON_VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) added. New config: { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } }
I20240501 14:09:11.125559 10881 ts_tablet_manager.cc:902] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:11.127272 10881 tablet_copy_client.cc:308] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:35221
I20240501 14:09:11.327746 8905 tablet_copy_service.cc:140] P f2e6587c4d054aa696aaa7ff76880a2f: Received BeginTabletCopySession request for tablet 96ab0c0bda54481b95b0fdf9d8da84df from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:33935)
I20240501 14:09:11.327912 8905 tablet_copy_service.cc:161] P f2e6587c4d054aa696aaa7ff76880a2f: Beginning new tablet copy session on tablet 96ab0c0bda54481b95b0fdf9d8da84df from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:33935: session id = 9a65600939334f6db8dcca7f07923ece-96ab0c0bda54481b95b0fdf9d8da84df
I20240501 14:09:11.329378 8905 tablet_copy_source_session.cc:205] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:11.332093 10881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ab0c0bda54481b95b0fdf9d8da84df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:11.334463 10881 tablet_copy_client.cc:783] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:11.334625 10881 tablet_copy_client.cc:647] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:11.345974 10881 tablet_copy_client.cc:523] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:11.347538 10881 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:11.394028 10905 ts_tablet_manager.cc:902] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:11.394691 10905 tablet_copy_client.cc:308] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:42941
I20240501 14:09:11.395992 9298 tablet_copy_service.cc:140] P 0c0d3d3aae054876a0a522a8325ddd02: Received BeginTabletCopySession request for tablet 7eac39a2ecdc40f0929ab332508189cd from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:47719)
I20240501 14:09:11.396102 9298 tablet_copy_service.cc:161] P 0c0d3d3aae054876a0a522a8325ddd02: Beginning new tablet copy session on tablet 7eac39a2ecdc40f0929ab332508189cd from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:47719: session id = 9a65600939334f6db8dcca7f07923ece-7eac39a2ecdc40f0929ab332508189cd
I20240501 14:09:11.397044 9298 tablet_copy_source_session.cc:205] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: Tablet Copy: opened 0 blocks and 1 log segments
W20240501 14:09:11.404101 8690 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.195:33719, user_credentials={real_user=slave}} blocked reactor thread for 46139.7us
I20240501 14:09:11.404382 10905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac39a2ecdc40f0929ab332508189cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:11.426344 10905 tablet_copy_client.cc:783] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:11.426743 10905 tablet_copy_client.cc:647] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:11.432111 10913 ts_tablet_manager.cc:902] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:11.433697 10905 tablet_copy_client.cc:523] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:11.436105 10905 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:11.437237 10913 tablet_copy_client.cc:308] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:42941
I20240501 14:09:11.441424 9298 tablet_copy_service.cc:140] P 0c0d3d3aae054876a0a522a8325ddd02: Received BeginTabletCopySession request for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:47719)
I20240501 14:09:11.441537 9298 tablet_copy_service.cc:161] P 0c0d3d3aae054876a0a522a8325ddd02: Beginning new tablet copy session on tablet 1b39b2a74a9c4dd78fc5541b3ce52772 from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:47719: session id = 9a65600939334f6db8dcca7f07923ece-1b39b2a74a9c4dd78fc5541b3ce52772
I20240501 14:09:11.442533 9298 tablet_copy_source_session.cc:205] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:11.443827 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b39b2a74a9c4dd78fc5541b3ce52772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:11.460789 10913 tablet_copy_client.cc:783] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:11.461133 10913 tablet_copy_client.cc:647] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:11.514739 10913 tablet_copy_client.cc:523] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:11.516350 10913 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:11.594142 10905 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=83 overwritten=0 applied=83 ignored=0} inserts{seen=768 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:11.594604 10905 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:11.594908 10905 ts_tablet_manager.cc:1367] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.159s user 0.081s sys 0.000s
I20240501 14:09:11.595506 10905 raft_consensus.cc:348] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:11.595759 10905 raft_consensus.cc:729] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:11.595930 10905 consensus_queue.cc:260] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 83, Last appended: 1.83, Last appended by leader: 83, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:11.596411 10905 ts_tablet_manager.cc:1398] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:11.598850 9298 tablet_copy_service.cc:342] P 0c0d3d3aae054876a0a522a8325ddd02: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-7eac39a2ecdc40f0929ab332508189cd received from {username='slave'} at 127.6.153.199:47719
I20240501 14:09:11.598934 9298 tablet_copy_service.cc:433] P 0c0d3d3aae054876a0a522a8325ddd02: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-7eac39a2ecdc40f0929ab332508189cd on tablet 7eac39a2ecdc40f0929ab332508189cd with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:11.600361 10905 ts_tablet_manager.cc:902] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: Initiating tablet copy from peer 25737230978048598f426b1a99b27441 (127.6.153.193:41195)
I20240501 14:09:11.600770 10905 tablet_copy_client.cc:308] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:41195
I20240501 14:09:11.601346 8774 tablet_copy_service.cc:140] P 25737230978048598f426b1a99b27441: Received BeginTabletCopySession request for tablet a6b7f72cb79d480a8a39e20c51d922f9 from peer 9a65600939334f6db8dcca7f07923ece ({username='slave'} at 127.6.153.199:48145)
I20240501 14:09:11.601449 8774 tablet_copy_service.cc:161] P 25737230978048598f426b1a99b27441: Beginning new tablet copy session on tablet a6b7f72cb79d480a8a39e20c51d922f9 from peer 9a65600939334f6db8dcca7f07923ece at {username='slave'} at 127.6.153.199:48145: session id = 9a65600939334f6db8dcca7f07923ece-a6b7f72cb79d480a8a39e20c51d922f9
I20240501 14:09:11.602422 8774 tablet_copy_source_session.cc:205] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:11.603793 10905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b7f72cb79d480a8a39e20c51d922f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:11.606185 10905 tablet_copy_client.cc:783] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:11.606324 10905 tablet_copy_client.cc:647] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:11.614152 10905 tablet_copy_client.cc:523] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:11.615862 10905 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: Bootstrap starting.
I20240501 14:09:11.977664 10244 raft_consensus.cc:1203] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.73->[1.74-1.87] Dedup: 1.83->[1.84-1.87]
I20240501 14:09:11.987164 10913 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=92 overwritten=0 applied=92 ignored=0} inserts{seen=764 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:11.995982 10913 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:11.996296 10913 ts_tablet_manager.cc:1367] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.480s user 0.089s sys 0.000s
I20240501 14:09:11.996868 10913 raft_consensus.cc:348] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:11.998162 10913 raft_consensus.cc:729] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:11.999433 10913 consensus_queue.cc:260] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 92, Last appended: 1.92, Last appended by leader: 92, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 84 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:12.000782 10913 ts_tablet_manager.cc:1398] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:09:12.001251 9298 tablet_copy_service.cc:342] P 0c0d3d3aae054876a0a522a8325ddd02: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-1b39b2a74a9c4dd78fc5541b3ce52772 received from {username='slave'} at 127.6.153.199:47719
I20240501 14:09:12.001332 9298 tablet_copy_service.cc:433] P 0c0d3d3aae054876a0a522a8325ddd02: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-1b39b2a74a9c4dd78fc5541b3ce52772 on tablet 1b39b2a74a9c4dd78fc5541b3ce52772 with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:12.002838 10241 raft_consensus.cc:1203] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.83->[1.84-1.99] Dedup: 1.92->[1.93-1.99]
I20240501 14:09:12.008744 10881 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=131 overwritten=0 applied=131 ignored=0} inserts{seen=803 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:12.009155 10881 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:12.009384 10881 ts_tablet_manager.cc:1367] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.662s user 0.111s sys 0.000s
I20240501 14:09:12.010566 10881 raft_consensus.cc:348] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:12.010782 10881 raft_consensus.cc:729] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:12.011899 10881 consensus_queue.cc:260] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 131, Last appended: 1.131, Last appended by leader: 131, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:12.019222 10881 ts_tablet_manager.cc:1398] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:09:12.019950 8905 tablet_copy_service.cc:342] P f2e6587c4d054aa696aaa7ff76880a2f: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-96ab0c0bda54481b95b0fdf9d8da84df received from {username='slave'} at 127.6.153.199:33935
I20240501 14:09:12.020054 8905 tablet_copy_service.cc:433] P f2e6587c4d054aa696aaa7ff76880a2f: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-96ab0c0bda54481b95b0fdf9d8da84df on tablet 96ab0c0bda54481b95b0fdf9d8da84df with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:12.036721 10905 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: Bootstrap replayed 1/1 log segments. Stats: ops{read=109 overwritten=0 applied=109 ignored=0} inserts{seen=781 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:12.037142 10905 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: Bootstrap complete.
I20240501 14:09:12.037396 10905 ts_tablet_manager.cc:1367] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: Time spent bootstrapping tablet: real 0.422s user 0.085s sys 0.012s
I20240501 14:09:12.038447 10905 raft_consensus.cc:348] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:12.038909 10905 raft_consensus.cc:729] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a65600939334f6db8dcca7f07923ece, State: Initialized, Role: LEARNER
I20240501 14:09:12.039064 10905 consensus_queue.cc:260] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 109, Last appended: 1.109, Last appended by leader: 109, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } }
I20240501 14:09:12.039628 10905 ts_tablet_manager.cc:1398] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:12.039677 10243 raft_consensus.cc:1203] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.98->[1.99-1.113] Dedup: 1.109->[1.110-1.113]
I20240501 14:09:12.041564 8774 tablet_copy_service.cc:342] P 25737230978048598f426b1a99b27441: Request end of tablet copy session 9a65600939334f6db8dcca7f07923ece-a6b7f72cb79d480a8a39e20c51d922f9 received from {username='slave'} at 127.6.153.199:48145
I20240501 14:09:12.041656 8774 tablet_copy_service.cc:433] P 25737230978048598f426b1a99b27441: ending tablet copy session 9a65600939334f6db8dcca7f07923ece-a6b7f72cb79d480a8a39e20c51d922f9 on tablet a6b7f72cb79d480a8a39e20c51d922f9 with peer 9a65600939334f6db8dcca7f07923ece
I20240501 14:09:12.060294 10632 raft_consensus.cc:1050] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:12.060778 10632 consensus_queue.cc:237] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 113, Committed index: 113, Last appended: 1.114, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:12.063227 9142 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 14:09:12.063958 10323 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:12.064697 10243 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 14:09:12.065075 10323 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:12.069505 9276 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 14:09:12.070626 10323 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:12.075206 10344 raft_consensus.cc:1050] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:12.075688 10344 consensus_queue.cc:237] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 99, Committed index: 99, Last appended: 1.100, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 101 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:12.077625 10241 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 100. Preceding OpId from leader: term: 1 index: 101. (index mismatch)
I20240501 14:09:12.078001 10650 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 101, Last known committed idx: 99, Time since last communication: 0.000s
I20240501 14:09:12.081297 8878 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 14:09:12.082000 10323 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:12.082139 9016 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 100. Preceding OpId from leader: term: 1 index: 101. (index mismatch)
I20240501 14:09:12.084486 10632 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.115: config changed from index 99 to 115, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.089012 10851 raft_consensus.cc:1050] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:12.089077 9142 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 99 to 115, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.089426 10851 consensus_queue.cc:237] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 88, Committed index: 88, Last appended: 1.88, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:12.092303 8878 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 99 to 115, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.093667 9145 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 14:09:12.093987 8606 catalog_manager.cc:5484] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 99 to 115, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:12.094144 9948 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 88, Time since last communication: 0.000s
I20240501 14:09:12.094724 10243 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 14:09:12.095192 9948 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 88, Time since last communication: 0.000s
I20240501 14:09:12.098183 10241 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 99 to 115, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.103987 10650 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 101, Last known committed idx: 99, Time since last communication: 0.000s
I20240501 14:09:12.106002 9015 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 14:09:12.107080 10650 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 88, Time since last communication: 0.000s
I20240501 14:09:12.108994 8754 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 100. Preceding OpId from leader: term: 1 index: 101. (index mismatch)
I20240501 14:09:12.109405 9948 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 101, Last known committed idx: 99, Time since last communication: 0.000s
I20240501 14:09:12.114001 9948 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.89: config changed from index 74 to 89, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.118815 9016 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 14:09:12.118703 9142 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index 74 to 89, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.120334 10411 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 14:09:12.121701 10241 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index 74 to 89, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.130237 9014 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index 74 to 89, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.130532 8604 catalog_manager.cc:5484] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 74 to 89, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:12.141840 10851 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.101: config changed from index 84 to 101, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 101 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.142684 9408 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 100. Preceding OpId from leader: term: 1 index: 101. (index mismatch)
I20240501 14:09:12.143483 9408 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 14:09:12.143833 8880 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 100. Preceding OpId from leader: term: 1 index: 101. (index mismatch)
I20240501 14:09:12.144398 8753 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.101: config changed from index 84 to 101, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 101 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.147182 9014 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.101: config changed from index 84 to 101, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 101 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.153096 10244 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.101: config changed from index 84 to 101, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 101 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.168445 9015 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 99 to 115, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.214270 9948 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 101, Last known committed idx: 99, Time since last communication: 0.000s
I20240501 14:09:12.216224 10650 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 101, Last known committed idx: 99, Time since last communication: 0.000s
I20240501 14:09:12.246922 8878 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 14:09:12.252995 9405 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.101: config changed from index 84 to 101, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 101 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.254926 9948 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 88, Time since last communication: 0.000s
I20240501 14:09:12.256469 8603 catalog_manager.cc:5484] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 reported cstate change: config changed from index 84 to 101, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 101 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.259040 9405 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index 74 to 89, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.262838 10344 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 14:09:12.263569 9276 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 99 to 115, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.293569 8878 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.101: config changed from index 84 to 101, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 101 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.302029 8881 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index 74 to 89, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.323451 10244 raft_consensus.cc:1203] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.113->[1.114-1.139] Dedup: 1.131->[1.132-1.139]
I20240501 14:09:12.332201 8754 consensus_queue.cc:237] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 122, Committed index: 122, Last appended: 1.122, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 123 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:12.333029 9277 consensus_queue.cc:237] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 103, Committed index: 103, Last appended: 1.104, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:12.334712 10242 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 104. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
I20240501 14:09:12.335126 10344 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 103, Time since last communication: 0.000s
I20240501 14:09:12.342228 8751 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 104. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
I20240501 14:09:12.343847 9948 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 103, Time since last communication: 0.000s
I20240501 14:09:12.349185 10344 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.105: config changed from index 101 to 105, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.350153 10243 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 122. Preceding OpId from leader: term: 1 index: 123. (index mismatch)
I20240501 14:09:12.350528 10488 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 123, Last known committed idx: 122, Time since last communication: 0.000s
I20240501 14:09:12.351387 10242 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 101 to 105, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.354166 8752 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 101 to 105, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.356900 8598 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 with cas_config_opid_index 101: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:12.357887 8608 catalog_manager.cc:5484] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 101 to 105, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:12.364760 8996 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 with delete_type TABLET_DATA_TOMBSTONED (TS 84c7d91c82524b69a9184fcf1949b5b0 not found in new config with opid_index 105) from {username='slave'} at 127.0.0.1:34080
I20240501 14:09:12.368939 9408 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 103. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
I20240501 14:09:12.378127 9014 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 122. Preceding OpId from leader: term: 1 index: 123. (index mismatch)
I20240501 14:09:12.379271 10632 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 123, Last known committed idx: 122, Time since last communication: 0.000s
I20240501 14:09:12.381224 10845 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.123: config changed from index 115 to 123, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 123 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.381649 9146 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 122. Preceding OpId from leader: term: 1 index: 123. (index mismatch)
I20240501 14:09:12.382062 10488 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 123, Last known committed idx: 122, Time since last communication: 0.000s
I20240501 14:09:12.382227 10650 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 103, Time since last communication: 0.000s
I20240501 14:09:12.383148 9146 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.123: config changed from index 115 to 123, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 123 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.384372 8597 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a6b7f72cb79d480a8a39e20c51d922f9 with cas_config_opid_index 115: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:12.385485 10242 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.123: config changed from index 115 to 123, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 123 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.385758 9408 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 101 to 105, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.387795 8608 catalog_manager.cc:5484] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a reported cstate change: config changed from index 115 to 123, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 123 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.388356 9276 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 122. Preceding OpId from leader: term: 1 index: 123. (index mismatch)
I20240501 14:09:12.390883 10322 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 123, Last known committed idx: 122, Time since last communication: 0.000s
I20240501 14:09:12.392096 8881 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 103. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
I20240501 14:09:12.402678 9276 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.123: config changed from index 115 to 123, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 123 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.405925 8863 tablet_service.cc:1514] Processing DeleteTablet for tablet a6b7f72cb79d480a8a39e20c51d922f9 with delete_type TABLET_DATA_TOMBSTONED (TS f2e6587c4d054aa696aaa7ff76880a2f not found in new config with opid_index 123) from {username='slave'} at 127.0.0.1:57374
I20240501 14:09:12.410300 10565 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 102, Time since last communication: 0.000s
I20240501 14:09:12.415880 8878 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 101 to 105, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.425240 9014 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.123: config changed from index 115 to 123, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 123 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
W20240501 14:09:12.435096 8687 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.198:32925, user_credentials={real_user=slave}} blocked reactor thread for 52495us
I20240501 14:09:12.443053 10980 tablet_replica.cc:330] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: stopping tablet replica
I20240501 14:09:12.443791 10980 raft_consensus.cc:2229] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:12.446480 10980 raft_consensus.cc:2258] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:12.450302 10980 ts_tablet_manager.cc:1875] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:12.456576 10978 tablet_replica.cc:330] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: stopping tablet replica
I20240501 14:09:12.456811 10978 raft_consensus.cc:2229] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:12.461525 10978 pending_rounds.cc:62] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: Trying to abort 1 pending ops.
I20240501 14:09:12.462019 10978 pending_rounds.cc:68] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: Aborting op as it isn't in flight: id { term: 1 index: 104 } timestamp: 7022889174303531008 op_type: WRITE_OP write_request { tablet_id: "1b39b2a74a9c4dd78fc5541b3ce52772" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\275\347\3120\227\001\356\365\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""a2946ac97bdd87b6" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022889174148018176 authz_token { token_data: "\010\254\234\311\261\006\"3\n\005slave\022*\n 091b87e33e224e50b05a4f016867ef06\020\001\030\001 \001(\001" signature: "<redacted>""<\036\021\270L\024D\246\374\3663\330rZ\327!=\320\337\021\325\003\346)bf\354\254\0361\354\331\376\371f3\277\343\005xDr\0279Z`\272\205\223\364\241\036`\025\375\212\346T\317\257\\z\331\253" signing_key_seq_num: 0 } } request_id { client_id: "c32512e56b3e4b6e83f81072c7610d1f" seq_no: 335 first_incomplete_seq_no: 335 attempt_no: 0 }
I20240501 14:09:12.462608 10978 raft_consensus.cc:2258] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:12.463563 10978 ts_tablet_manager.cc:1875] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:12.469466 10980 ts_tablet_manager.cc:1888] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.122
I20240501 14:09:12.469640 10980 log.cc:1196] T a6b7f72cb79d480a8a39e20c51d922f9 P f2e6587c4d054aa696aaa7ff76880a2f: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/a6b7f72cb79d480a8a39e20c51d922f9
I20240501 14:09:12.470506 8596 catalog_manager.cc:4830] TS f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221): tablet a6b7f72cb79d480a8a39e20c51d922f9 (table rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]) successfully deleted
I20240501 14:09:12.473650 9277 consensus_queue.cc:237] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 90, Committed index: 90, Last appended: 1.90, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 91 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:12.474543 10978 ts_tablet_manager.cc:1888] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.104
I20240501 14:09:12.475278 10243 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 90. Preceding OpId from leader: term: 1 index: 91. (index mismatch)
I20240501 14:09:12.475718 9948 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 91, Last known committed idx: 90, Time since last communication: 0.000s
I20240501 14:09:12.486793 10978 log.cc:1196] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 84c7d91c82524b69a9184fcf1949b5b0: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/1b39b2a74a9c4dd78fc5541b3ce52772
I20240501 14:09:12.492368 9405 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 90. Preceding OpId from leader: term: 1 index: 91. (index mismatch)
I20240501 14:09:12.493074 8599 catalog_manager.cc:4830] TS 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719): tablet 1b39b2a74a9c4dd78fc5541b3ce52772 (table rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]) successfully deleted
I20240501 14:09:12.500283 9015 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 90. Preceding OpId from leader: term: 1 index: 91. (index mismatch)
I20240501 14:09:12.514446 8881 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 90. Preceding OpId from leader: term: 1 index: 91. (index mismatch)
I20240501 14:09:12.515605 9948 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 91, Last known committed idx: 90, Time since last communication: 0.000s
I20240501 14:09:12.544107 10650 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 91, Last known committed idx: 90, Time since last communication: 0.000s
I20240501 14:09:12.547235 10650 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 91, Last known committed idx: 90, Time since last communication: 0.000s
I20240501 14:09:12.554973 10650 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.91: config changed from index 89 to 91, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 91 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.557089 10243 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.91: config changed from index 89 to 91, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 91 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.560521 8608 catalog_manager.cc:5484] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece reported cstate change: config changed from index 89 to 91, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 91 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.563498 9126 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eac39a2ecdc40f0929ab332508189cd with delete_type TABLET_DATA_TOMBSTONED (TS 73ea0b7a5e4c45a3814d46e74d62331a not found in new config with opid_index 91) from {username='slave'} at 127.0.0.1:49360
I20240501 14:09:12.565989 9014 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.91: config changed from index 89 to 91, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 91 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.569591 8598 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7eac39a2ecdc40f0929ab332508189cd with cas_config_opid_index 89: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:12.604809 9407 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.91: config changed from index 89 to 91, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 91 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.659601 8885 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.91: config changed from index 89 to 91, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 91 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.708492 10987 tablet_replica.cc:330] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: stopping tablet replica
I20240501 14:09:12.708657 10987 raft_consensus.cc:2229] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:12.708787 10987 raft_consensus.cc:2258] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Raft consensus is shut down!
W20240501 14:09:12.708900 9211 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.198:32925, user_credentials={real_user=slave}} blocked reactor thread for 52306.2us
I20240501 14:09:12.711915 10416 raft_consensus.cc:1050] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: attempting to promote NON_VOTER 9a65600939334f6db8dcca7f07923ece to VOTER
I20240501 14:09:12.712435 10416 consensus_queue.cc:237] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 146, Committed index: 146, Last appended: 1.147, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 148 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:12.715185 10987 ts_tablet_manager.cc:1875] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:12.715487 10241 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 144. Preceding OpId from leader: term: 1 index: 148. (index mismatch)
I20240501 14:09:12.715545 9407 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 147. Preceding OpId from leader: term: 1 index: 148. (index mismatch)
I20240501 14:09:12.715939 10007 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 148, Last known committed idx: 146, Time since last communication: 0.000s
I20240501 14:09:12.718987 10987 ts_tablet_manager.cc:1888] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.90
I20240501 14:09:12.719947 10086 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 148, Last known committed idx: 143, Time since last communication: 0.000s
I20240501 14:09:12.721146 10241 raft_consensus.cc:1203] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 LEARNER]: Deduplicated request from leader. Original: 1.144->[1.145-1.148] Dedup: 1.147->[1.148-1.148]
I20240501 14:09:12.721588 10987 log.cc:1196] T 7eac39a2ecdc40f0929ab332508189cd P 73ea0b7a5e4c45a3814d46e74d62331a: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/7eac39a2ecdc40f0929ab332508189cd
I20240501 14:09:12.722391 8598 catalog_manager.cc:4830] TS 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983): tablet 7eac39a2ecdc40f0929ab332508189cd (table rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]) successfully deleted
I20240501 14:09:12.725917 8754 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 144. Preceding OpId from leader: term: 1 index: 148. (index mismatch)
I20240501 14:09:12.728770 9277 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 147. Preceding OpId from leader: term: 1 index: 148. (index mismatch)
I20240501 14:09:12.729475 9142 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 147. Preceding OpId from leader: term: 1 index: 148. (index mismatch)
I20240501 14:09:12.749047 10007 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 148, Last known committed idx: 146, Time since last communication: 0.000s
I20240501 14:09:12.756665 10007 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 148, Last known committed idx: 146, Time since last communication: 0.000s
I20240501 14:09:12.764922 10007 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 148, Last known committed idx: 144, Time since last communication: 0.000s
I20240501 14:09:12.773965 10007 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 LEADER]: Committing config change with OpId 1.148: config changed from index 114 to 148, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 148 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.776407 9405 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.148: config changed from index 114 to 148, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 148 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.778007 8606 catalog_manager.cc:5484] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: config changed from index 114 to 148, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: 148 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:12.781262 10242 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.148: config changed from index 114 to 148, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 148 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.791129 9146 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.148: config changed from index 114 to 148, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 148 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.792980 8878 consensus_queue.cc:237] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 148, Committed index: 148, Last appended: 1.148, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 149 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } }
I20240501 14:09:12.796842 9407 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 148. Preceding OpId from leader: term: 1 index: 149. (index mismatch)
I20240501 14:09:12.796931 10240 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 148. Preceding OpId from leader: term: 1 index: 149. (index mismatch)
I20240501 14:09:12.797227 10416 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 149, Last known committed idx: 148, Time since last communication: 0.000s
I20240501 14:09:12.797618 10416 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 149, Last known committed idx: 148, Time since last communication: 0.000s
I20240501 14:09:12.804132 10240 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.149: config changed from index 148 to 149, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 149 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.804236 11000 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 LEADER]: Committing config change with OpId 1.149: config changed from index 148 to 149, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 149 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.808523 8608 catalog_manager.cc:5484] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece reported cstate change: config changed from index 148 to 149, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: 149 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.812229 8732 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ab0c0bda54481b95b0fdf9d8da84df with delete_type TABLET_DATA_TOMBSTONED (TS 25737230978048598f426b1a99b27441 not found in new config with opid_index 149) from {username='slave'} at 127.0.0.1:57504
I20240501 14:09:12.822530 9277 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 148. Preceding OpId from leader: term: 1 index: 149. (index mismatch)
I20240501 14:09:12.823558 9274 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.148: config changed from index 114 to 148, 9a65600939334f6db8dcca7f07923ece (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 148 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.829571 9145 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 148. Preceding OpId from leader: term: 1 index: 149. (index mismatch)
I20240501 14:09:12.852996 11002 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 149, Last known committed idx: 146, Time since last communication: 0.000s
I20240501 14:09:12.853359 11002 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 149, Last known committed idx: 148, Time since last communication: 0.000s
I20240501 14:09:12.859899 8596 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 96ab0c0bda54481b95b0fdf9d8da84df with cas_config_opid_index 148: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:12.861533 9407 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.149: config changed from index 148 to 149, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 149 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.875628 9146 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.149: config changed from index 148 to 149, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 149 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:12.886129 9274 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.149: config changed from index 148 to 149, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 149 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } }
I20240501 14:09:13.030356 11005 tablet_replica.cc:330] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: stopping tablet replica
I20240501 14:09:13.031224 11005 raft_consensus.cc:2229] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:13.042743 11005 pending_rounds.cc:62] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Trying to abort 2 pending ops.
I20240501 14:09:13.042946 11005 pending_rounds.cc:68] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Aborting op as it isn't in flight: id { term: 1 index: 148 } timestamp: 7022889175909081088 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" old_config { opid_index: 114 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: true } } } new_config { opid_index: 148 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } } }
I20240501 14:09:13.043356 11005 raft_consensus.cc:2894] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Aborting config change with OpId 1.148: Aborted: Op aborted
I20240501 14:09:13.043502 11005 raft_consensus.cc:2866] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240501 14:09:13.043641 11005 pending_rounds.cc:68] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Aborting op as it isn't in flight: id { term: 1 index: 147 } timestamp: 7022889175838834688 op_type: WRITE_OP write_request { tablet_id: "96ab0c0bda54481b95b0fdf9d8da84df" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\267yh;\204\210\270\317\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""223d754761bd33c1" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022889175797387264 authz_token { token_data: "\010\254\234\311\261\006\"3\n\005slave\022*\n 51c5f5c435c04dcea37a412861300245\020\001\030\001 \001(\001" signature: "<redacted>""}\251\370\320~\266\213m=7\366\364\222\275\264\223h\177\324\220{\243,\230\322\376\251A\356L\252\231\212\357\204;\236|lz\261\274g\214Ha\010\255\016+\225@n\2760\2641\373*\273\000\024pF" signing_key_seq_num: 0 } } request_id { client_id: "85d7b6151ebf485dbd55d9f921d536c3" seq_no: 385 first_incomplete_seq_no: 385 attempt_no: 0 }
I20240501 14:09:13.046964 11005 raft_consensus.cc:2258] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:13.058171 11005 ts_tablet_manager.cc:1875] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:13.062825 11005 ts_tablet_manager.cc:1888] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.148
I20240501 14:09:13.062938 11005 log.cc:1196] T 96ab0c0bda54481b95b0fdf9d8da84df P 25737230978048598f426b1a99b27441: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/96ab0c0bda54481b95b0fdf9d8da84df
I20240501 14:09:13.063603 8597 catalog_manager.cc:4830] TS 25737230978048598f426b1a99b27441 (127.6.153.193:41195): tablet 96ab0c0bda54481b95b0fdf9d8da84df (table rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]) successfully deleted
W20240501 14:09:13.654281 8820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.196:34983, user_credentials={real_user=slave}} blocked reactor thread for 47185.7us
I20240501 14:09:14.167438 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:14.212078 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:14.234282 8996 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:14.241362 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:14.252382 9248 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:14.271921 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:14.355163 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:14.781298 9274 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 136, Committed index: 136, Last appended: 1.136, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } }
I20240501 14:09:14.784240 10240 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240501 14:09:14.785040 9948 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 136, Time since last communication: 0.000s
I20240501 14:09:14.785188 9142 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240501 14:09:14.785418 9408 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240501 14:09:14.785557 9948 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 136, Time since last communication: 0.000s
I20240501 14:09:14.785687 9016 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240501 14:09:14.785755 10851 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 136, Time since last communication: 0.000s
I20240501 14:09:14.786965 10650 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 136, Time since last communication: 0.000s
I20240501 14:09:14.788388 10650 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.137: config changed from index 59 to 137, NON_VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) added. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } } }
I20240501 14:09:14.789690 9012 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.137: config changed from index 59 to 137, NON_VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) added. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } } }
I20240501 14:09:14.791108 9147 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.137: config changed from index 59 to 137, NON_VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) added. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } } }
I20240501 14:09:14.792191 8608 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 59 to 137, NON_VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) added. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:14.793607 10242 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.137: config changed from index 59 to 137, NON_VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) added. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } } }
I20240501 14:09:14.813305 9408 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.137: config changed from index 59 to 137, NON_VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) added. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } } }
W20240501 14:09:14.863509 9212 consensus_peers.cc:485] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 -> Peer 25737230978048598f426b1a99b27441 (127.6.153.193:41195): Couldn't send request to peer 25737230978048598f426b1a99b27441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d86cde53ceb4b63842ea9ec09659318. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:15.308305 11117 ts_tablet_manager.cc:902] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: Initiating tablet copy from peer 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:15.309232 11117 tablet_copy_client.cc:308] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:42941
I20240501 14:09:15.310931 9298 tablet_copy_service.cc:140] P 0c0d3d3aae054876a0a522a8325ddd02: Received BeginTabletCopySession request for tablet 2d86cde53ceb4b63842ea9ec09659318 from peer 25737230978048598f426b1a99b27441 ({username='slave'} at 127.6.153.193:51181)
I20240501 14:09:15.311039 9298 tablet_copy_service.cc:161] P 0c0d3d3aae054876a0a522a8325ddd02: Beginning new tablet copy session on tablet 2d86cde53ceb4b63842ea9ec09659318 from peer 25737230978048598f426b1a99b27441 at {username='slave'} at 127.6.153.193:51181: session id = 25737230978048598f426b1a99b27441-2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:15.312018 9298 tablet_copy_source_session.cc:205] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:15.321615 11117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d86cde53ceb4b63842ea9ec09659318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:15.324487 11117 tablet_copy_client.cc:783] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:15.324769 11117 tablet_copy_client.cc:647] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:15.406064 11117 tablet_copy_client.cc:523] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:15.408126 11117 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: Bootstrap starting.
I20240501 14:09:15.892648 11117 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: Bootstrap replayed 1/1 log segments. Stats: ops{read=144 overwritten=0 applied=144 ignored=0} inserts{seen=772 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:15.893139 11117 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: Bootstrap complete.
I20240501 14:09:15.893438 11117 ts_tablet_manager.cc:1367] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: Time spent bootstrapping tablet: real 0.485s user 0.122s sys 0.007s
I20240501 14:09:15.894279 11117 raft_consensus.cc:348] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } }
I20240501 14:09:15.894522 11117 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 25737230978048598f426b1a99b27441, State: Initialized, Role: LEARNER
I20240501 14:09:15.894701 11117 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 144, Last appended: 1.144, Last appended by leader: 144, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: NON_VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: true } }
I20240501 14:09:15.895371 11117 ts_tablet_manager.cc:1398] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:15.897730 9298 tablet_copy_service.cc:342] P 0c0d3d3aae054876a0a522a8325ddd02: Request end of tablet copy session 25737230978048598f426b1a99b27441-2d86cde53ceb4b63842ea9ec09659318 received from {username='slave'} at 127.6.153.193:51181
I20240501 14:09:15.897821 9298 tablet_copy_service.cc:433] P 0c0d3d3aae054876a0a522a8325ddd02: ending tablet copy session 25737230978048598f426b1a99b27441-2d86cde53ceb4b63842ea9ec09659318 on tablet 2d86cde53ceb4b63842ea9ec09659318 with peer 25737230978048598f426b1a99b27441
I20240501 14:09:15.954782 8752 raft_consensus.cc:1203] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.136->[1.137-1.152] Dedup: 1.144->[1.145-1.152]
I20240501 14:09:16.171294 11124 raft_consensus.cc:1050] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: attempting to promote NON_VOTER 25737230978048598f426b1a99b27441 to VOTER
I20240501 14:09:16.171736 11124 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 153, Committed index: 153, Last appended: 1.154, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } }
I20240501 14:09:16.174902 10241 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 153. Preceding OpId from leader: term: 1 index: 155. (index mismatch)
I20240501 14:09:16.174966 8753 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 1 LEARNER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 154. Preceding OpId from leader: term: 1 index: 155. (index mismatch)
I20240501 14:09:16.175279 9015 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 154. Preceding OpId from leader: term: 1 index: 155. (index mismatch)
I20240501 14:09:16.175843 11123 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 155, Last known committed idx: 153, Time since last communication: 0.000s
I20240501 14:09:16.176192 11123 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 155, Last known committed idx: 153, Time since last communication: 0.000s
I20240501 14:09:16.176465 11123 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 155, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 14:09:16.177175 9147 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 154. Preceding OpId from leader: term: 1 index: 155. (index mismatch)
I20240501 14:09:16.177902 10650 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 155, Last known committed idx: 153, Time since last communication: 0.000s
I20240501 14:09:16.181149 10674 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.155: config changed from index 137 to 155, 25737230978048598f426b1a99b27441 (127.6.153.193) changed from NON_VOTER to VOTER. New config: { opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.183450 9015 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.155: config changed from index 137 to 155, 25737230978048598f426b1a99b27441 (127.6.153.193) changed from NON_VOTER to VOTER. New config: { opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.186097 8606 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index 137 to 155, 25737230978048598f426b1a99b27441 (127.6.153.193) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.188848 8753 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.155: config changed from index 137 to 155, 25737230978048598f426b1a99b27441 (127.6.153.193) changed from NON_VOTER to VOTER. New config: { opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.191949 10242 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.155: config changed from index 137 to 155, 25737230978048598f426b1a99b27441 (127.6.153.193) changed from NON_VOTER to VOTER. New config: { opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.198693 9147 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.155: config changed from index 137 to 155, 25737230978048598f426b1a99b27441 (127.6.153.193) changed from NON_VOTER to VOTER. New config: { opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
W20240501 14:09:16.227949 8687 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.198:32925, user_credentials={real_user=slave}} blocked reactor thread for 51506.2us
I20240501 14:09:16.251272 9405 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 153. Preceding OpId from leader: term: 1 index: 155. (index mismatch)
I20240501 14:09:16.291133 10650 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 155, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 14:09:16.299111 9409 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.155: config changed from index 137 to 155, 25737230978048598f426b1a99b27441 (127.6.153.193) changed from NON_VOTER to VOTER. New config: { opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.464568 9277 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318"
dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:42642
I20240501 14:09:16.464854 9277 raft_consensus.cc:595] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Received request to transfer leadership
W20240501 14:09:16.469341 9223 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54122: Service unavailable: leader transfer in progress
I20240501 14:09:16.470475 10650 raft_consensus.cc:979] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: : Instructing follower 84c7d91c82524b69a9184fcf1949b5b0 to start an election
I20240501 14:09:16.470568 10650 raft_consensus.cc:1067] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Signalling peer 84c7d91c82524b69a9184fcf1949b5b0 to start an election
I20240501 14:09:16.471421 9015 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318"
dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
from {username='slave'} at 127.6.153.197:48271
I20240501 14:09:16.471575 9015 raft_consensus.cc:482] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:16.471632 9015 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:16.472751 9015 raft_consensus.cc:504] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } }
I20240501 14:09:16.473299 9015 leader_election.cc:288] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 2 election: Requested vote from peers 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105), 25737230978048598f426b1a99b27441 (127.6.153.193:41195)
I20240501 14:09:16.474239 10237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 159 } } ignore_live_leader: true dest_uuid: "9a65600939334f6db8dcca7f07923ece"
I20240501 14:09:16.474438 10237 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:16.475440 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 159 } } ignore_live_leader: true dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:16.475559 9409 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:16.475684 10237 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 2.
I20240501 14:09:16.476653 9409 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 2.
W20240501 14:09:16.478875 9223 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54122: Service unavailable: leader transfer in progress
I20240501 14:09:16.488754 9142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 159 } } ignore_live_leader: true dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:16.489004 9142 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:16.490238 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 159 } } ignore_live_leader: true dest_uuid: "0c0d3d3aae054876a0a522a8325ddd02"
I20240501 14:09:16.490365 9278 raft_consensus.cc:3032] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:16.490406 9278 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c0d3d3aae054876a0a522a8325ddd02, State: Running, Role: LEADER
I20240501 14:09:16.490388 9142 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 2.
I20240501 14:09:16.490585 9278 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 159, Committed index: 159, Last appended: 1.159, Last appended by leader: 159, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } }
I20240501 14:09:16.490825 9278 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Advancing to term 2
W20240501 14:09:16.490926 9223 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54122: Illegal state: replica 0c0d3d3aae054876a0a522a8325ddd02 is not leader of this config: current role FOLLOWER
I20240501 14:09:16.492612 9278 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 2.
I20240501 14:09:16.501338 8951 leader_election.cc:302] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 6 voters: 4 yes votes; 0 no votes. yes voters: 0c0d3d3aae054876a0a522a8325ddd02, 6b961a167ae64dbc9d102ce5ddf721f3, 84c7d91c82524b69a9184fcf1949b5b0, 9a65600939334f6db8dcca7f07923ece; no voters:
I20240501 14:09:16.501670 10414 raft_consensus.cc:2781] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:16.501760 10414 raft_consensus.cc:686] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 LEADER]: Becoming Leader. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Running, Role: LEADER
I20240501 14:09:16.501889 10414 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 159, Committed index: 159, Last appended: 1.159, Last appended by leader: 159, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } }
I20240501 14:09:16.504570 8609 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: term changed from 1 to 2, leader changed from 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) to 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195). New cstate: current_term: 2 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 155 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:16.507321 8751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d86cde53ceb4b63842ea9ec09659318" candidate_uuid: "84c7d91c82524b69a9184fcf1949b5b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 159 } } ignore_live_leader: true dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:16.507462 8751 raft_consensus.cc:3037] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:16.508592 8751 raft_consensus.cc:2454] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c7d91c82524b69a9184fcf1949b5b0 in term 2.
I20240501 14:09:16.617003 9142 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 159. Preceding OpId from leader: term: 2 index: 161. (index mismatch)
I20240501 14:09:16.620196 9278 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 159. Preceding OpId from leader: term: 2 index: 161. (index mismatch)
I20240501 14:09:16.628846 8753 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 159. Preceding OpId from leader: term: 2 index: 161. (index mismatch)
I20240501 14:09:16.630460 10414 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 160, Last known committed idx: 158, Time since last communication: 0.000s
I20240501 14:09:16.652334 11180 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 160, Last known committed idx: 159, Time since last communication: 0.000s
I20240501 14:09:16.682672 10542 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 160, Last known committed idx: 159, Time since last communication: 0.000s
I20240501 14:09:16.731527 10242 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 159. Preceding OpId from leader: term: 2 index: 161. (index mismatch)
I20240501 14:09:16.731823 9406 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 159. Preceding OpId from leader: term: 2 index: 161. (index mismatch)
I20240501 14:09:16.734594 11180 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 160, Last known committed idx: 158, Time since last communication: 0.000s
I20240501 14:09:16.750092 10864 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 160, Last known committed idx: 159, Time since last communication: 0.000s
W20240501 14:09:16.769932 8820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.196:34983, user_credentials={real_user=slave}} blocked reactor thread for 43926.5us
I20240501 14:09:16.805470 9015 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 161, Committed index: 161, Last appended: 2.162, Last appended by leader: 159, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 163 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } }
I20240501 14:09:16.808288 10242 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 162. Preceding OpId from leader: term: 2 index: 163. (index mismatch)
I20240501 14:09:16.808775 9406 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 162. Preceding OpId from leader: term: 2 index: 163. (index mismatch)
I20240501 14:09:16.808959 11180 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 163, Last known committed idx: 161, Time since last communication: 0.000s
I20240501 14:09:16.809583 11180 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 163, Last known committed idx: 161, Time since last communication: 0.000s
I20240501 14:09:16.812040 9145 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 162. Preceding OpId from leader: term: 2 index: 163. (index mismatch)
I20240501 14:09:16.813469 8751 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 162. Preceding OpId from leader: term: 2 index: 163. (index mismatch)
I20240501 14:09:16.814002 10542 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 LEADER]: Committing config change with OpId 2.163: config changed from index 155 to 163, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 163 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.814414 10864 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 163, Last known committed idx: 161, Time since last communication: 0.000s
I20240501 14:09:16.815281 9409 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Committing config change with OpId 2.163: config changed from index 155 to 163, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 163 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.817121 8607 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index 155 to 163, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New cstate: current_term: 2 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 163 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:16.817348 10241 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Committing config change with OpId 2.163: config changed from index 155 to 163, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 163 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.821056 8751 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Committing config change with OpId 2.163: config changed from index 155 to 163, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 163 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.823038 9248 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d86cde53ceb4b63842ea9ec09659318 with delete_type TABLET_DATA_TOMBSTONED (TS 0c0d3d3aae054876a0a522a8325ddd02 not found in new config with opid_index 163) from {username='slave'} at 127.0.0.1:54040
I20240501 14:09:16.825857 10414 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 163, Last known committed idx: 161, Time since last communication: 0.000s
I20240501 14:09:16.828868 8599 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2d86cde53ceb4b63842ea9ec09659318 with cas_config_opid_index 155: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:16.831934 9144 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Committing config change with OpId 2.163: config changed from index 155 to 163, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 163 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } }
I20240501 14:09:16.905041 11188 tablet_replica.cc:330] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: stopping tablet replica
I20240501 14:09:16.917119 11188 raft_consensus.cc:2229] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:16.917943 11188 raft_consensus.cc:2258] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:16.920882 11188 ts_tablet_manager.cc:1875] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:16.970932 11188 ts_tablet_manager.cc:1888] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.161
I20240501 14:09:16.971156 11188 log.cc:1196] T 2d86cde53ceb4b63842ea9ec09659318 P 0c0d3d3aae054876a0a522a8325ddd02: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:16.971838 8598 catalog_manager.cc:4830] TS 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941): tablet 2d86cde53ceb4b63842ea9ec09659318 (table rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]) successfully deleted
I20240501 14:09:17.425189 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:17.434937 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:17.468806 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240501 14:09:17.472492 8951 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.197:42941, user_credentials={real_user=slave}} blocked reactor thread for 52252.7us
I20240501 14:09:17.475596 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
W20240501 14:09:17.491966 8820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.196:34983, user_credentials={real_user=slave}} blocked reactor thread for 61607.9us
I20240501 14:09:17.570739 8996 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:09:17.599188 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:17.607393 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:18.352666 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20240501 14:09:18.353928 8687 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.198:32925, user_credentials={real_user=slave}} blocked reactor thread for 44351.3us
I20240501 14:09:18.392854 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:18.413954 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:18.442852 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:18.443184 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:18.444506 8996 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:09:18.484928 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240501 14:09:19.391702 8949 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.194:35221, user_credentials={real_user=slave}} blocked reactor thread for 62463us
W20240501 14:09:19.441361 8951 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.196:34983, user_credentials={real_user=slave}} blocked reactor thread for 67937.5us
W20240501 14:09:19.653370 8949 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.198:32925, user_credentials={real_user=slave}} blocked reactor thread for 47902.9us
W20240501 14:09:20.689916 8951 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.196:34983, user_credentials={real_user=slave}} blocked reactor thread for 66281.8us
W20240501 14:09:20.690448 8820 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.153.196:34983, user_credentials={real_user=slave}} blocked reactor thread for 66857.7us
I20240501 14:09:20.905553 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:20.925825 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:20.931980 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:20.942368 8996 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:09:20.964274 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:20.970096 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:20.983922 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:22.100124 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:22.100430 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:22.166270 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:22.204718 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:22.249086 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:22.362588 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:22.407882 8996 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:09:23.067488 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:23.067687 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:23.072016 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:23.077059 8996 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:09:23.084164 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:23.092540 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:23.098057 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
9fa87eb4295445af9eb74cc9bbb89994 | 127.6.153.254:42171 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.6.153.212:42933 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
0c0d3d3aae054876a0a522a8325ddd02 | 127.6.153.197:42941 | HEALTHY | <none> | 3 | 0
25737230978048598f426b1a99b27441 | 127.6.153.193:41195 | HEALTHY | <none> | 3 | 0
6b961a167ae64dbc9d102ce5ddf721f3 | 127.6.153.198:32925 | HEALTHY | <none> | 1 | 0
73ea0b7a5e4c45a3814d46e74d62331a | 127.6.153.196:34983 | HEALTHY | <none> | 2 | 0
84c7d91c82524b69a9184fcf1949b5b0 | 127.6.153.195:33719 | HEALTHY | <none> | 5 | 0
9a65600939334f6db8dcca7f07923ece | 127.6.153.199:42105 | HEALTHY | <none> | 0 | 0
f2e6587c4d054aa696aaa7ff76880a2f | 127.6.153.194:35221 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.6.153.193 | experimental | 127.6.153.193:41195
local_ip_for_outbound_sockets | 127.6.153.194 | experimental | 127.6.153.194:35221
local_ip_for_outbound_sockets | 127.6.153.195 | experimental | 127.6.153.195:33719
local_ip_for_outbound_sockets | 127.6.153.196 | experimental | 127.6.153.196:34983
local_ip_for_outbound_sockets | 127.6.153.197 | experimental | 127.6.153.197:42941
local_ip_for_outbound_sockets | 127.6.153.198 | experimental | 127.6.153.198:32925
local_ip_for_outbound_sockets | 127.6.153.199 | experimental | 127.6.153.199:42105
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.6.153.193:41195
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.153.194:35221
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.153.195:33719
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.153.196:34983
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.153.197:42941
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.153.198:32925
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.6.153.199:42105
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.6.153.212:42933 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 11
First Quartile | 11
Median | 11
Third Quartile | 12
Maximum | 12
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 14:09:23.126426 6759 log_verifier.cc:126] Checking tablet 188101b7f3784e48975a673952b2bc63
I20240501 14:09:23.287845 6759 log_verifier.cc:177] Verified matching terms for 349 ops in tablet 188101b7f3784e48975a673952b2bc63
I20240501 14:09:23.288095 6759 log_verifier.cc:126] Checking tablet 1b39b2a74a9c4dd78fc5541b3ce52772
I20240501 14:09:23.426849 6759 log_verifier.cc:177] Verified matching terms for 288 ops in tablet 1b39b2a74a9c4dd78fc5541b3ce52772
I20240501 14:09:23.427104 6759 log_verifier.cc:126] Checking tablet 203c4d95b91f495f91f96b9308d1efc3
I20240501 14:09:23.562641 6759 log_verifier.cc:177] Verified matching terms for 255 ops in tablet 203c4d95b91f495f91f96b9308d1efc3
I20240501 14:09:23.562839 6759 log_verifier.cc:126] Checking tablet 2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:23.694427 6759 log_verifier.cc:177] Verified matching terms for 243 ops in tablet 2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:23.694607 6759 log_verifier.cc:126] Checking tablet 30ef391ceec44fd4b7195b41d7732843
I20240501 14:09:23.828887 6759 log_verifier.cc:177] Verified matching terms for 290 ops in tablet 30ef391ceec44fd4b7195b41d7732843
I20240501 14:09:23.829067 6759 log_verifier.cc:126] Checking tablet 4323f538490b4bf99fa7f96f80c8a88f
I20240501 14:09:23.960541 6759 log_verifier.cc:177] Verified matching terms for 279 ops in tablet 4323f538490b4bf99fa7f96f80c8a88f
I20240501 14:09:23.960745 6759 log_verifier.cc:126] Checking tablet 4b2fe0f030204ecca740b14dfd6605ff
I20240501 14:09:24.098709 6759 log_verifier.cc:177] Verified matching terms for 292 ops in tablet 4b2fe0f030204ecca740b14dfd6605ff
I20240501 14:09:24.098901 6759 log_verifier.cc:126] Checking tablet 560c845cf74c4df1bf33732abc84620d
I20240501 14:09:24.237143 6759 log_verifier.cc:177] Verified matching terms for 293 ops in tablet 560c845cf74c4df1bf33732abc84620d
I20240501 14:09:24.237335 6759 log_verifier.cc:126] Checking tablet 7eac39a2ecdc40f0929ab332508189cd
I20240501 14:09:24.357829 6759 log_verifier.cc:177] Verified matching terms for 251 ops in tablet 7eac39a2ecdc40f0929ab332508189cd
I20240501 14:09:24.358008 6759 log_verifier.cc:126] Checking tablet 7f9e94ea8d834320a06db3087864a08c
I20240501 14:09:24.490597 6759 log_verifier.cc:177] Verified matching terms for 280 ops in tablet 7f9e94ea8d834320a06db3087864a08c
I20240501 14:09:24.490784 6759 log_verifier.cc:126] Checking tablet 858bf448fd5249f0a86d17d6d011a6b4
I20240501 14:09:24.625185 6759 log_verifier.cc:177] Verified matching terms for 282 ops in tablet 858bf448fd5249f0a86d17d6d011a6b4
I20240501 14:09:24.625356 6759 log_verifier.cc:126] Checking tablet 96ab0c0bda54481b95b0fdf9d8da84df
I20240501 14:09:24.758494 6759 log_verifier.cc:177] Verified matching terms for 298 ops in tablet 96ab0c0bda54481b95b0fdf9d8da84df
I20240501 14:09:24.758674 6759 log_verifier.cc:126] Checking tablet a6b7f72cb79d480a8a39e20c51d922f9
I20240501 14:09:24.889349 6759 log_verifier.cc:177] Verified matching terms for 286 ops in tablet a6b7f72cb79d480a8a39e20c51d922f9
I20240501 14:09:24.889532 6759 log_verifier.cc:126] Checking tablet ab008693c1db49e493c3fac38646c8c0
I20240501 14:09:24.922752 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet ab008693c1db49e493c3fac38646c8c0
I20240501 14:09:24.922816 6759 log_verifier.cc:126] Checking tablet d21fe5dd65fe4fbea85fecc8ce19bc43
I20240501 14:09:25.083070 6759 log_verifier.cc:177] Verified matching terms for 377 ops in tablet d21fe5dd65fe4fbea85fecc8ce19bc43
I20240501 14:09:25.083302 6759 log_verifier.cc:126] Checking tablet e72446fcffed439c8653937330e5b8ee
I20240501 14:09:25.236191 6759 log_verifier.cc:177] Verified matching terms for 348 ops in tablet e72446fcffed439c8653937330e5b8ee
I20240501 14:09:25.238090 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.200:0
--local_ip_for_outbound_sockets=127.6.153.200
--webserver_interface=127.6.153.200
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--builtin_ntp_servers=127.6.153.212:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:09:25.344924 11493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:25.345163 11493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:25.345217 11493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:25.349321 11493 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:25.349390 11493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:25.349475 11493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.200
I20240501 14:09:25.354012 11493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.200:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.6.153.200
--webserver_port=0
--tserver_master_addrs=127.6.153.254:42171
--never_fsync=true
--heap_profile_path=/tmp/kudu.11493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.200
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:25.354781 11493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:25.355820 11493 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:25.364006 11499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:25.364476 11493 server_base.cc:1034] running on GCE node
W20240501 14:09:25.364710 11501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:25.365029 11498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:25.365339 11493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:25.365813 11493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:25.366972 11493 hybrid_clock.cc:648] HybridClock initialized: now 1714572565366947 us; error 41 us; skew 500 ppm
I20240501 14:09:25.368685 11493 webserver.cc:460] Webserver started at http://127.6.153.200:42891/ using document root <none> and password file <none>
I20240501 14:09:25.369150 11493 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:25.369197 11493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:25.369354 11493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:25.370918 11493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "95a4031ef42440ebbdb822d7fb315062"
format_stamp: "Formatted at 2024-05-01 14:09:25 on dist-test-slave-cmf0"
I20240501 14:09:25.371412 11493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "95a4031ef42440ebbdb822d7fb315062"
format_stamp: "Formatted at 2024-05-01 14:09:25 on dist-test-slave-cmf0"
I20240501 14:09:25.374320 11493 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:25.376475 11507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:25.377403 11493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:25.377491 11493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "95a4031ef42440ebbdb822d7fb315062"
format_stamp: "Formatted at 2024-05-01 14:09:25 on dist-test-slave-cmf0"
I20240501 14:09:25.377573 11493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:25.395618 11493 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:25.396245 11493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:25.396387 11493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:25.396899 11493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:25.397890 11493 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:25.397938 11493 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:25.397998 11493 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:25.398028 11493 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:25.406757 11493 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.200:39129
I20240501 14:09:25.406777 11622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.200:39129 every 8 connection(s)
I20240501 14:09:25.407670 11493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 14:09:25.414031 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 11493
I20240501 14:09:25.414129 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 14:09:25.418027 11623 heartbeater.cc:347] Connected to a master server at 127.6.153.254:42171
I20240501 14:09:25.418234 11623 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:25.418740 11623 heartbeater.cc:511] Master 127.6.153.254:42171 requested a full tablet report, sending...
I20240501 14:09:25.419665 8609 ts_manager.cc:194] Registered new tserver with Master: 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129)
I20240501 14:09:25.420224 8609 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.200:42283
I20240501 14:09:25.606747 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:25.615414 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:25.618582 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:25.624431 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:25.627252 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:25.628335 11556 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:25.630051 8996 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:09:25.638105 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:25.737257 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:25.742769 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:25.746295 11556 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:25.746821 8996 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:09:25.748062 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:25.754423 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:25.760617 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:25.767297 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:25.818167 9014 consensus_queue.cc:237] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 290, Committed index: 290, Last appended: 1.290, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:25.819941 8753 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240501 14:09:25.820235 8885 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240501 14:09:25.820371 9401 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240501 14:09:25.820418 11719 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240501 14:09:25.820677 11720 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240501 14:09:25.820981 9278 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240501 14:09:25.821614 11721 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240501 14:09:25.821903 11721 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240501 14:09:25.824115 11721 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.291: config changed from index -1 to 291, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
W20240501 14:09:25.825935 8949 consensus_peers.cc:485] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 30ef391ceec44fd4b7195b41d7732843. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:25.826467 9401 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index -1 to 291, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.826671 8885 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index -1 to 291, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.827525 8603 catalog_manager.cc:5484] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index -1 to 291, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:25.829200 9278 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index -1 to 291, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.829190 8753 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index -1 to 291, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.840030 9278 consensus_queue.cc:237] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 251, Committed index: 251, Last appended: 1.251, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:25.842181 9401 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 251. Preceding OpId from leader: term: 1 index: 252. (index mismatch)
I20240501 14:09:25.842828 10241 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 251. Preceding OpId from leader: term: 1 index: 252. (index mismatch)
I20240501 14:09:25.843377 9014 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 251. Preceding OpId from leader: term: 1 index: 252. (index mismatch)
I20240501 14:09:25.843778 11487 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 252, Last known committed idx: 251, Time since last communication: 0.000s
I20240501 14:09:25.844136 11487 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 252, Last known committed idx: 251, Time since last communication: 0.000s
I20240501 14:09:25.844306 11737 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 252, Last known committed idx: 251, Time since last communication: 0.000s
I20240501 14:09:25.844615 8885 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 251. Preceding OpId from leader: term: 1 index: 252. (index mismatch)
I20240501 14:09:25.846832 11740 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 252, Last known committed idx: 251, Time since last communication: 0.000s
W20240501 14:09:25.847045 9211 consensus_peers.cc:485] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7eac39a2ecdc40f0929ab332508189cd. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:25.847759 11740 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.252: config changed from index 91 to 252, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.849615 10241 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.252: config changed from index 91 to 252, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.850028 8612 catalog_manager.cc:5484] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 91 to 252, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:25.852046 9401 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.252: config changed from index 91 to 252, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.853765 9014 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.252: config changed from index 91 to 252, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.857136 8885 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.252: config changed from index 91 to 252, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.862960 9401 consensus_queue.cc:237] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 292, Committed index: 292, Last appended: 1.292, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:25.864746 10241 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240501 14:09:25.865110 8753 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240501 14:09:25.865186 8885 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240501 14:09:25.865520 10482 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240501 14:09:25.865689 11753 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240501 14:09:25.865901 9144 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240501 14:09:25.866003 10482 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240501 14:09:25.866422 10482 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240501 14:09:25.868428 11755 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 LEADER]: Committing config change with OpId 1.293: config changed from index 74 to 293, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.868780 9144 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 74 to 293, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.868780 10241 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 74 to 293, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.870780 8885 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 74 to 293, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.871145 8608 catalog_manager.cc:5484] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 reported cstate change: config changed from index 74 to 293, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" committed_config { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:25.871464 8753 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 74 to 293, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
W20240501 14:09:25.874147 9342 consensus_peers.cc:485] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b2fe0f030204ecca740b14dfd6605ff. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:25.881608 9278 consensus_queue.cc:237] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 349, Committed index: 349, Last appended: 1.349, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
W20240501 14:09:25.883441 9211 consensus_peers.cc:485] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 188101b7f3784e48975a673952b2bc63. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:25.883473 8885 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 349. Preceding OpId from leader: term: 1 index: 350. (index mismatch)
I20240501 14:09:25.883538 9014 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 349. Preceding OpId from leader: term: 1 index: 350. (index mismatch)
I20240501 14:09:25.883886 9401 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 349. Preceding OpId from leader: term: 1 index: 350. (index mismatch)
I20240501 14:09:25.883960 11739 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 350, Last known committed idx: 349, Time since last communication: 0.000s
I20240501 14:09:25.884060 8753 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 349. Preceding OpId from leader: term: 1 index: 350. (index mismatch)
I20240501 14:09:25.884296 11739 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 350, Last known committed idx: 349, Time since last communication: 0.000s
I20240501 14:09:25.884454 11740 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 350, Last known committed idx: 349, Time since last communication: 0.000s
I20240501 14:09:25.884747 11487 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 350, Last known committed idx: 349, Time since last communication: 0.000s
I20240501 14:09:25.887123 11739 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.350: config changed from index -1 to 350, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.887463 9401 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.350: config changed from index -1 to 350, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.887516 8753 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.350: config changed from index -1 to 350, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.889880 8608 catalog_manager.cc:5484] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index -1 to 350, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.890604 8885 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.350: config changed from index -1 to 350, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.896939 9014 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.350: config changed from index -1 to 350, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.902629 8885 consensus_queue.cc:237] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 298, Committed index: 298, Last appended: 1.298, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:25.904558 10241 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 298. Preceding OpId from leader: term: 1 index: 299. (index mismatch)
I20240501 14:09:25.904974 11777 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 299, Last known committed idx: 298, Time since last communication: 0.000s
I20240501 14:09:25.905102 9144 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 298. Preceding OpId from leader: term: 1 index: 299. (index mismatch)
I20240501 14:09:25.905103 9401 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 298. Preceding OpId from leader: term: 1 index: 299. (index mismatch)
I20240501 14:09:25.905390 9278 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 298. Preceding OpId from leader: term: 1 index: 299. (index mismatch)
I20240501 14:09:25.905907 11778 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 299, Last known committed idx: 298, Time since last communication: 0.000s
I20240501 14:09:25.906147 11780 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 299, Last known committed idx: 298, Time since last communication: 0.000s
I20240501 14:09:25.907107 11779 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 299, Last known committed idx: 298, Time since last communication: 0.000s
I20240501 14:09:25.910750 11434 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 LEADER]: Committing config change with OpId 1.299: config changed from index 149 to 299, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
W20240501 14:09:25.911810 8818 consensus_peers.cc:485] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96ab0c0bda54481b95b0fdf9d8da84df. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:25.912781 9278 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.299: config changed from index 149 to 299, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.913159 8608 catalog_manager.cc:5484] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: config changed from index 149 to 299, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:25.916293 10241 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.299: config changed from index 149 to 299, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.920809 9144 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.299: config changed from index 149 to 299, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:25.923579 9401 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.299: config changed from index 149 to 299, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:26.350420 11841 ts_tablet_manager.cc:902] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:26.351100 11841 tablet_copy_client.cc:308] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:42941
I20240501 14:09:26.355137 9298 tablet_copy_service.cc:140] P 0c0d3d3aae054876a0a522a8325ddd02: Received BeginTabletCopySession request for tablet 7eac39a2ecdc40f0929ab332508189cd from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:49577)
I20240501 14:09:26.355230 9298 tablet_copy_service.cc:161] P 0c0d3d3aae054876a0a522a8325ddd02: Beginning new tablet copy session on tablet 7eac39a2ecdc40f0929ab332508189cd from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:49577: session id = 95a4031ef42440ebbdb822d7fb315062-7eac39a2ecdc40f0929ab332508189cd
I20240501 14:09:26.355947 9298 tablet_copy_source_session.cc:205] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:26.359164 11841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac39a2ecdc40f0929ab332508189cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:26.364004 11841 tablet_copy_client.cc:783] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:26.364159 11841 tablet_copy_client.cc:647] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:26.365839 11841 tablet_copy_client.cc:523] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:26.367209 11841 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:26.386642 11841 log.cc:824] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:26.421473 11873 ts_tablet_manager.cc:902] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221)
I20240501 14:09:26.421945 11873 tablet_copy_client.cc:308] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:35221
I20240501 14:09:26.422175 11623 heartbeater.cc:503] Master 127.6.153.254:42171 was elected leader, sending a full tablet report...
I20240501 14:09:26.426259 8905 tablet_copy_service.cc:140] P f2e6587c4d054aa696aaa7ff76880a2f: Received BeginTabletCopySession request for tablet 96ab0c0bda54481b95b0fdf9d8da84df from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:50825)
I20240501 14:09:26.426358 8905 tablet_copy_service.cc:161] P f2e6587c4d054aa696aaa7ff76880a2f: Beginning new tablet copy session on tablet 96ab0c0bda54481b95b0fdf9d8da84df from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:50825: session id = 95a4031ef42440ebbdb822d7fb315062-96ab0c0bda54481b95b0fdf9d8da84df
I20240501 14:09:26.427206 8905 tablet_copy_source_session.cc:205] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:26.428118 11873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ab0c0bda54481b95b0fdf9d8da84df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:26.430164 11873 tablet_copy_client.cc:783] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:26.430289 11873 tablet_copy_client.cc:647] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:26.433337 11873 tablet_copy_client.cc:523] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:26.434240 11873 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:26.443384 11878 ts_tablet_manager.cc:902] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719)
I20240501 14:09:26.443990 11878 tablet_copy_client.cc:308] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:33719
I20240501 14:09:26.447980 9036 tablet_copy_service.cc:140] P 84c7d91c82524b69a9184fcf1949b5b0: Received BeginTabletCopySession request for tablet 30ef391ceec44fd4b7195b41d7732843 from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:43055)
I20240501 14:09:26.448081 9036 tablet_copy_service.cc:161] P 84c7d91c82524b69a9184fcf1949b5b0: Beginning new tablet copy session on tablet 30ef391ceec44fd4b7195b41d7732843 from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:43055: session id = 95a4031ef42440ebbdb822d7fb315062-30ef391ceec44fd4b7195b41d7732843
I20240501 14:09:26.448966 9036 tablet_copy_source_session.cc:205] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:26.449803 11878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef391ceec44fd4b7195b41d7732843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:26.451817 11878 tablet_copy_client.cc:783] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:26.451952 11878 tablet_copy_client.cc:647] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:26.454202 11878 tablet_copy_client.cc:523] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:26.455243 11878 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:26.459600 11884 ts_tablet_manager.cc:902] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:26.460119 11884 tablet_copy_client.cc:308] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:42941
I20240501 14:09:26.460592 9298 tablet_copy_service.cc:140] P 0c0d3d3aae054876a0a522a8325ddd02: Received BeginTabletCopySession request for tablet 188101b7f3784e48975a673952b2bc63 from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:49577)
I20240501 14:09:26.460692 9298 tablet_copy_service.cc:161] P 0c0d3d3aae054876a0a522a8325ddd02: Beginning new tablet copy session on tablet 188101b7f3784e48975a673952b2bc63 from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:49577: session id = 95a4031ef42440ebbdb822d7fb315062-188101b7f3784e48975a673952b2bc63
I20240501 14:09:26.461562 9298 tablet_copy_source_session.cc:205] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:26.462373 11884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188101b7f3784e48975a673952b2bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:26.464375 11884 tablet_copy_client.cc:783] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:26.464558 11884 tablet_copy_client.cc:647] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:26.466326 11884 tablet_copy_client.cc:523] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:26.467450 11884 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:26.472642 11889 ts_tablet_manager.cc:902] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925)
I20240501 14:09:26.473141 11889 tablet_copy_client.cc:308] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:32925
I20240501 14:09:26.478886 9429 tablet_copy_service.cc:140] P 6b961a167ae64dbc9d102ce5ddf721f3: Received BeginTabletCopySession request for tablet 4b2fe0f030204ecca740b14dfd6605ff from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:54061)
I20240501 14:09:26.478981 9429 tablet_copy_service.cc:161] P 6b961a167ae64dbc9d102ce5ddf721f3: Beginning new tablet copy session on tablet 4b2fe0f030204ecca740b14dfd6605ff from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:54061: session id = 95a4031ef42440ebbdb822d7fb315062-4b2fe0f030204ecca740b14dfd6605ff
I20240501 14:09:26.479931 9429 tablet_copy_source_session.cc:205] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:26.480984 11889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2fe0f030204ecca740b14dfd6605ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:26.485755 11889 tablet_copy_client.cc:783] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:26.485932 11889 tablet_copy_client.cc:647] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:26.487813 11889 tablet_copy_client.cc:523] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:26.488863 11889 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:26.553246 11841 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=252 overwritten=0 applied=252 ignored=0} inserts{seen=934 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:26.553712 11841 tablet_bootstrap.cc:492] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:26.554164 11841 ts_tablet_manager.cc:1367] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.187s user 0.175s sys 0.012s
I20240501 14:09:26.556835 11841 raft_consensus.cc:348] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.557205 11841 raft_consensus.cc:729] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:26.557610 11841 consensus_queue.cc:260] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 252, Last appended: 1.252, Last appended by leader: 252, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 252 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.558651 11841 ts_tablet_manager.cc:1398] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20240501 14:09:26.560765 9298 tablet_copy_service.cc:342] P 0c0d3d3aae054876a0a522a8325ddd02: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-7eac39a2ecdc40f0929ab332508189cd received from {username='slave'} at 127.6.153.200:49577
I20240501 14:09:26.560861 9298 tablet_copy_service.cc:433] P 0c0d3d3aae054876a0a522a8325ddd02: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-7eac39a2ecdc40f0929ab332508189cd on tablet 7eac39a2ecdc40f0929ab332508189cd with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:26.682423 11873 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=299 overwritten=0 applied=299 ignored=0} inserts{seen=968 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:26.682756 11873 tablet_bootstrap.cc:492] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:26.683068 11873 ts_tablet_manager.cc:1367] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.249s user 0.192s sys 0.025s
I20240501 14:09:26.683717 11873 raft_consensus.cc:348] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.683889 11873 raft_consensus.cc:729] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:26.684034 11873 consensus_queue.cc:260] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 299, Last appended: 1.299, Last appended by leader: 299, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.684366 11873 ts_tablet_manager.cc:1398] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:26.684850 8905 tablet_copy_service.cc:342] P f2e6587c4d054aa696aaa7ff76880a2f: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-96ab0c0bda54481b95b0fdf9d8da84df received from {username='slave'} at 127.6.153.200:50825
I20240501 14:09:26.684950 8905 tablet_copy_service.cc:433] P f2e6587c4d054aa696aaa7ff76880a2f: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-96ab0c0bda54481b95b0fdf9d8da84df on tablet 96ab0c0bda54481b95b0fdf9d8da84df with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:26.741957 11878 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=291 overwritten=0 applied=291 ignored=0} inserts{seen=922 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:26.742305 11878 tablet_bootstrap.cc:492] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:26.742551 11878 ts_tablet_manager.cc:1367] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.287s user 0.191s sys 0.019s
I20240501 14:09:26.743047 11878 raft_consensus.cc:348] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.743191 11878 raft_consensus.cc:729] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:26.743358 11878 consensus_queue.cc:260] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 291, Last appended: 1.291, Last appended by leader: 291, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.743793 11878 ts_tablet_manager.cc:1398] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:09:26.744556 9036 tablet_copy_service.cc:342] P 84c7d91c82524b69a9184fcf1949b5b0: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-30ef391ceec44fd4b7195b41d7732843 received from {username='slave'} at 127.6.153.200:43055
I20240501 14:09:26.744653 9036 tablet_copy_service.cc:433] P 84c7d91c82524b69a9184fcf1949b5b0: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-30ef391ceec44fd4b7195b41d7732843 on tablet 30ef391ceec44fd4b7195b41d7732843 with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:26.769202 11889 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=293 overwritten=0 applied=293 ignored=0} inserts{seen=921 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:26.769471 11889 tablet_bootstrap.cc:492] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:26.769651 11889 ts_tablet_manager.cc:1367] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.281s user 0.189s sys 0.008s
I20240501 14:09:26.770005 11889 raft_consensus.cc:348] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.770107 11889 raft_consensus.cc:729] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:26.770184 11889 consensus_queue.cc:260] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 293, Last appended: 1.293, Last appended by leader: 293, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.770452 11889 ts_tablet_manager.cc:1398] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:26.770931 9429 tablet_copy_service.cc:342] P 6b961a167ae64dbc9d102ce5ddf721f3: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-4b2fe0f030204ecca740b14dfd6605ff received from {username='slave'} at 127.6.153.200:54061
I20240501 14:09:26.771006 9429 tablet_copy_service.cc:433] P 6b961a167ae64dbc9d102ce5ddf721f3: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-4b2fe0f030204ecca740b14dfd6605ff on tablet 4b2fe0f030204ecca740b14dfd6605ff with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:26.777949 11884 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=350 overwritten=0 applied=350 ignored=0} inserts{seen=981 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:26.778201 11884 tablet_bootstrap.cc:492] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:26.778373 11884 ts_tablet_manager.cc:1367] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.311s user 0.225s sys 0.021s
I20240501 14:09:26.778730 11884 raft_consensus.cc:348] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.778821 11884 raft_consensus.cc:729] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:26.778915 11884 consensus_queue.cc:260] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 350, Last appended: 1.350, Last appended by leader: 350, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 350 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:26.779160 11884 ts_tablet_manager.cc:1398] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:26.779633 9298 tablet_copy_service.cc:342] P 0c0d3d3aae054876a0a522a8325ddd02: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-188101b7f3784e48975a673952b2bc63 received from {username='slave'} at 127.6.153.200:49577
I20240501 14:09:26.779714 9298 tablet_copy_service.cc:433] P 0c0d3d3aae054876a0a522a8325ddd02: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-188101b7f3784e48975a673952b2bc63 on tablet 188101b7f3784e48975a673952b2bc63 with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:26.803902 11576 raft_consensus.cc:1203] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.251->[1.252-1.252] Dedup: 1.252->[]
I20240501 14:09:26.818367 11576 raft_consensus.cc:1203] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.290->[1.291-1.291] Dedup: 1.291->[]
I20240501 14:09:26.853746 11576 raft_consensus.cc:1203] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.298->[1.299-1.299] Dedup: 1.299->[]
I20240501 14:09:26.897420 11576 raft_consensus.cc:1203] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.349->[1.350-1.350] Dedup: 1.350->[]
I20240501 14:09:26.950400 11576 raft_consensus.cc:1203] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.292->[1.293-1.293] Dedup: 1.293->[]
I20240501 14:09:27.242623 11927 raft_consensus.cc:1050] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:27.243079 11927 consensus_queue.cc:237] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 299, Committed index: 299, Last appended: 1.299, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.244702 9278 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 299. Preceding OpId from leader: term: 1 index: 300. (index mismatch)
I20240501 14:09:27.244783 11576 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 299. Preceding OpId from leader: term: 1 index: 300. (index mismatch)
I20240501 14:09:27.244807 9401 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 299. Preceding OpId from leader: term: 1 index: 300. (index mismatch)
I20240501 14:09:27.245208 10241 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 299. Preceding OpId from leader: term: 1 index: 300. (index mismatch)
I20240501 14:09:27.245296 11985 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 300, Last known committed idx: 299, Time since last communication: 0.000s
I20240501 14:09:27.245566 11927 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 300, Last known committed idx: 299, Time since last communication: 0.000s
I20240501 14:09:27.245795 11985 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 300, Last known committed idx: 299, Time since last communication: 0.000s
I20240501 14:09:27.245923 11927 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 300, Last known committed idx: 299, Time since last communication: 0.000s
I20240501 14:09:27.246008 9144 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 299. Preceding OpId from leader: term: 1 index: 300. (index mismatch)
I20240501 14:09:27.247138 11984 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 300, Last known committed idx: 299, Time since last communication: 0.000s
I20240501 14:09:27.248955 11928 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 LEADER]: Committing config change with OpId 1.300: config changed from index 299 to 300, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.249428 9144 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.300: config changed from index 299 to 300, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.250355 10241 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.300: config changed from index 299 to 300, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.251575 9409 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.300: config changed from index 299 to 300, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.251509 11576 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.300: config changed from index 299 to 300, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.253378 8609 catalog_manager.cc:5484] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece reported cstate change: config changed from index 299 to 300, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.254719 9277 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.300: config changed from index 299 to 300, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.257515 8885 consensus_queue.cc:237] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 300, Committed index: 300, Last appended: 1.300, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 301 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.258705 11576 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 300. Preceding OpId from leader: term: 1 index: 301. (index mismatch)
I20240501 14:09:27.258781 10241 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 300. Preceding OpId from leader: term: 1 index: 301. (index mismatch)
I20240501 14:09:27.258785 9277 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 300. Preceding OpId from leader: term: 1 index: 301. (index mismatch)
I20240501 14:09:27.258814 9409 raft_consensus.cc:1261] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 300. Preceding OpId from leader: term: 1 index: 301. (index mismatch)
I20240501 14:09:27.259131 11928 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 301, Last known committed idx: 300, Time since last communication: 0.000s
I20240501 14:09:27.259287 11786 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 301, Last known committed idx: 300, Time since last communication: 0.000s
I20240501 14:09:27.259449 11927 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 301, Last known committed idx: 300, Time since last communication: 0.000s
I20240501 14:09:27.259678 11984 consensus_queue.cc:1035] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 301, Last known committed idx: 300, Time since last communication: 0.000s
I20240501 14:09:27.261286 11786 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P f2e6587c4d054aa696aaa7ff76880a2f [term 1 LEADER]: Committing config change with OpId 1.301: config changed from index 300 to 301, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 301 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.261574 9409 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.301: config changed from index 300 to 301, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 301 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.262444 9277 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.301: config changed from index 300 to 301, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 301 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.263486 8596 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 96ab0c0bda54481b95b0fdf9d8da84df with cas_config_opid_index 300: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:27.263775 8606 catalog_manager.cc:5484] T 96ab0c0bda54481b95b0fdf9d8da84df P 6b961a167ae64dbc9d102ce5ddf721f3 reported cstate change: config changed from index 300 to 301, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: 301 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.264161 10241 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.301: config changed from index 300 to 301, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 301 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.266219 11576 raft_consensus.cc:2932] T 96ab0c0bda54481b95b0fdf9d8da84df P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.301: config changed from index 300 to 301, VOTER 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196) evicted. New config: { opid_index: 301 OBSOLETE_local: false peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.266521 9126 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ab0c0bda54481b95b0fdf9d8da84df with delete_type TABLET_DATA_TOMBSTONED (TS 73ea0b7a5e4c45a3814d46e74d62331a not found in new config with opid_index 301) from {username='slave'} at 127.0.0.1:49360
I20240501 14:09:27.267117 11996 tablet_replica.cc:330] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: stopping tablet replica
I20240501 14:09:27.267242 11996 raft_consensus.cc:2229] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:27.267431 11996 raft_consensus.cc:2258] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:27.269073 11996 ts_tablet_manager.cc:1875] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:27.270983 11996 ts_tablet_manager.cc:1888] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.300
I20240501 14:09:27.271059 11996 log.cc:1196] T 96ab0c0bda54481b95b0fdf9d8da84df P 73ea0b7a5e4c45a3814d46e74d62331a: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/96ab0c0bda54481b95b0fdf9d8da84df
I20240501 14:09:27.271553 8598 catalog_manager.cc:4830] TS 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983): tablet 96ab0c0bda54481b95b0fdf9d8da84df (table rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]) successfully deleted
I20240501 14:09:27.277987 11923 raft_consensus.cc:1050] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:27.278364 11923 consensus_queue.cc:237] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 252, Committed index: 252, Last appended: 1.252, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 253 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.279778 11576 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 252. Preceding OpId from leader: term: 1 index: 253. (index mismatch)
I20240501 14:09:27.279827 9409 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 252. Preceding OpId from leader: term: 1 index: 253. (index mismatch)
I20240501 14:09:27.279978 9014 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 252. Preceding OpId from leader: term: 1 index: 253. (index mismatch)
I20240501 14:09:27.280021 10241 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 252. Preceding OpId from leader: term: 1 index: 253. (index mismatch)
I20240501 14:09:27.280128 8885 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 252. Preceding OpId from leader: term: 1 index: 253. (index mismatch)
I20240501 14:09:27.280246 11998 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 253, Last known committed idx: 252, Time since last communication: 0.000s
I20240501 14:09:27.280388 11923 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 253, Last known committed idx: 252, Time since last communication: 0.000s
I20240501 14:09:27.280519 11999 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 253, Last known committed idx: 252, Time since last communication: 0.000s
I20240501 14:09:27.280741 11738 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 253, Last known committed idx: 252, Time since last communication: 0.000s
I20240501 14:09:27.280937 12000 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 253, Last known committed idx: 252, Time since last communication: 0.000s
I20240501 14:09:27.283401 11998 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.253: config changed from index 252 to 253, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 253 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.283864 9409 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.253: config changed from index 252 to 253, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 253 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.284204 10241 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.253: config changed from index 252 to 253, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 253 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.285930 8885 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.253: config changed from index 252 to 253, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 253 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.286093 8603 catalog_manager.cc:5484] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 252 to 253, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 253 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:27.288733 11576 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.253: config changed from index 252 to 253, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 253 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
W20240501 14:09:27.290537 11924 consensus_queue.cc:1159] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "95a4031ef42440ebbdb822d7fb315062" responder_term: 1 status { last_received { term: 1 index: 253 } last_committed_idx: 253 last_received_current_leader { term: 1 index: 253 } }
I20240501 14:09:27.290367 9014 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.253: config changed from index 252 to 253, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 253 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.290803 9277 consensus_queue.cc:237] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 253, Committed index: 253, Last appended: 1.253, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.292237 10241 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 14:09:27.292743 12000 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 14:09:27.292871 9409 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 14:09:27.293388 12000 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 14:09:27.293498 11576 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 14:09:27.294344 8885 raft_consensus.cc:1261] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 14:09:27.294450 11924 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 14:09:27.295135 11924 consensus_queue.cc:1035] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 14:09:27.295845 12000 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.254: config changed from index 253 to 254, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.296667 11576 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index 253 to 254, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.297578 8598 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7eac39a2ecdc40f0929ab332508189cd with cas_config_opid_index 253: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:27.297922 9409 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index 253 to 254, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.298058 10241 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index 253 to 254, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.298389 8608 catalog_manager.cc:5484] T 7eac39a2ecdc40f0929ab332508189cd P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 253 to 254, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:27.299170 8885 raft_consensus.cc:2932] T 7eac39a2ecdc40f0929ab332508189cd P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index 253 to 254, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.301054 8996 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eac39a2ecdc40f0929ab332508189cd with delete_type TABLET_DATA_TOMBSTONED (TS 84c7d91c82524b69a9184fcf1949b5b0 not found in new config with opid_index 254) from {username='slave'} at 127.0.0.1:34080
I20240501 14:09:27.301434 11926 raft_consensus.cc:1050] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:27.301483 12006 tablet_replica.cc:330] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: stopping tablet replica
I20240501 14:09:27.301654 12006 raft_consensus.cc:2229] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:27.301812 12006 raft_consensus.cc:2258] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:27.301846 11926 consensus_queue.cc:237] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 291, Committed index: 291, Last appended: 1.291, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.303674 8753 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240501 14:09:27.303720 9409 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240501 14:09:27.304122 11576 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240501 14:09:27.304175 11925 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240501 14:09:27.304221 9277 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240501 14:09:27.304396 11926 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240501 14:09:27.304534 11721 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240501 14:09:27.304677 8884 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240501 14:09:27.305198 11721 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240501 14:09:27.305481 11721 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240501 14:09:27.305583 12006 ts_tablet_manager.cc:1875] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:27.307817 12006 ts_tablet_manager.cc:1888] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.253
I20240501 14:09:27.307897 12006 log.cc:1196] T 7eac39a2ecdc40f0929ab332508189cd P 84c7d91c82524b69a9184fcf1949b5b0: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/7eac39a2ecdc40f0929ab332508189cd
I20240501 14:09:27.308660 8599 catalog_manager.cc:4830] TS 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719): tablet 7eac39a2ecdc40f0929ab332508189cd (table rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]) successfully deleted
I20240501 14:09:27.308871 11926 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.309352 8884 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.309937 11576 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.310664 9277 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.310789 9409 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.311477 8753 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.312158 8606 catalog_manager.cc:5484] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 reported cstate change: config changed from index 291 to 292, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } attrs { replace: true } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.316509 9014 consensus_queue.cc:237] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 292, Committed index: 292, Last appended: 1.292, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.317580 11576 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240501 14:09:27.317618 9409 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240501 14:09:27.317672 8753 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240501 14:09:27.317920 12008 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240501 14:09:27.318070 12009 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240501 14:09:27.318284 9277 raft_consensus.cc:1261] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240501 14:09:27.318450 11925 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240501 14:09:27.318593 12008 consensus_queue.cc:1035] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240501 14:09:27.320151 11926 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.320561 8753 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.320780 9277 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.321431 8599 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 30ef391ceec44fd4b7195b41d7732843 with cas_config_opid_index 292: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:27.322760 8603 catalog_manager.cc:5484] T 30ef391ceec44fd4b7195b41d7732843 P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index 292 to 293, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:27.323386 9409 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.323484 11576 raft_consensus.cc:2932] T 30ef391ceec44fd4b7195b41d7732843 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.326038 8863 tablet_service.cc:1514] Processing DeleteTablet for tablet 30ef391ceec44fd4b7195b41d7732843 with delete_type TABLET_DATA_TOMBSTONED (TS f2e6587c4d054aa696aaa7ff76880a2f not found in new config with opid_index 293) from {username='slave'} at 127.0.0.1:57374
I20240501 14:09:27.327026 12015 tablet_replica.cc:330] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: stopping tablet replica
I20240501 14:09:27.327195 12015 raft_consensus.cc:2229] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:27.327385 12015 raft_consensus.cc:2258] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:27.328567 12015 ts_tablet_manager.cc:1875] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:27.330475 12015 ts_tablet_manager.cc:1888] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.292
I20240501 14:09:27.330577 12015 log.cc:1196] T 30ef391ceec44fd4b7195b41d7732843 P f2e6587c4d054aa696aaa7ff76880a2f: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/30ef391ceec44fd4b7195b41d7732843
I20240501 14:09:27.331167 8596 catalog_manager.cc:4830] TS f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194:35221): tablet 30ef391ceec44fd4b7195b41d7732843 (table rebalance_test_table_4 [id=51c5f5c435c04dcea37a412861300245]) successfully deleted
I20240501 14:09:27.366740 9014 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 243, Committed index: 243, Last appended: 2.243, Last appended by leader: 159, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:27.368211 8753 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 243. Preceding OpId from leader: term: 2 index: 244. (index mismatch)
I20240501 14:09:27.368328 10241 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 243. Preceding OpId from leader: term: 2 index: 244. (index mismatch)
I20240501 14:09:27.368433 9409 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 243. Preceding OpId from leader: term: 2 index: 244. (index mismatch)
I20240501 14:09:27.368562 11926 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 244, Last known committed idx: 243, Time since last communication: 0.000s
I20240501 14:09:27.368777 12008 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 244, Last known committed idx: 243, Time since last communication: 0.000s
W20240501 14:09:27.368942 8949 consensus_peers.cc:485] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d86cde53ceb4b63842ea9ec09659318. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:27.369117 9144 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 243. Preceding OpId from leader: term: 2 index: 244. (index mismatch)
I20240501 14:09:27.369469 12008 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 244, Last known committed idx: 243, Time since last communication: 0.000s
I20240501 14:09:27.369978 12008 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 244, Last known committed idx: 243, Time since last communication: 0.000s
I20240501 14:09:27.370754 11926 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 LEADER]: Committing config change with OpId 2.244: config changed from index 163 to 244, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.371209 8753 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Committing config change with OpId 2.244: config changed from index 163 to 244, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.372181 10241 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Committing config change with OpId 2.244: config changed from index 163 to 244, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.373488 9409 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Committing config change with OpId 2.244: config changed from index 163 to 244, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.373673 8608 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index 163 to 244, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 2 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:27.377574 9144 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Committing config change with OpId 2.244: config changed from index 163 to 244, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.382411 8753 consensus_queue.cc:237] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 286, Committed index: 286, Last appended: 1.286, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:27.384385 10241 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 286. Preceding OpId from leader: term: 1 index: 287. (index mismatch)
I20240501 14:09:27.384699 9014 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 286. Preceding OpId from leader: term: 1 index: 287. (index mismatch)
I20240501 14:09:27.384771 12059 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 287, Last known committed idx: 286, Time since last communication: 0.000s
I20240501 14:09:27.385156 9144 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 286. Preceding OpId from leader: term: 1 index: 287. (index mismatch)
I20240501 14:09:27.385208 9277 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 286. Preceding OpId from leader: term: 1 index: 287. (index mismatch)
I20240501 14:09:27.385607 12058 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 287, Last known committed idx: 286, Time since last communication: 0.000s
I20240501 14:09:27.386324 12058 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 287, Last known committed idx: 286, Time since last communication: 0.000s
I20240501 14:09:27.386634 12058 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 287, Last known committed idx: 286, Time since last communication: 0.000s
I20240501 14:09:27.387871 12058 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.287: config changed from index 123 to 287, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
W20240501 14:09:27.389585 8687 consensus_peers.cc:485] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6b7f72cb79d480a8a39e20c51d922f9. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:27.389673 9277 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.287: config changed from index 123 to 287, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.389809 9144 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.287: config changed from index 123 to 287, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.390058 9014 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.287: config changed from index 123 to 287, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.391554 8612 catalog_manager.cc:5484] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 123 to 287, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:27.392380 10241 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.287: config changed from index 123 to 287, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.399442 9277 consensus_queue.cc:237] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 288, Committed index: 288, Last appended: 1.288, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:27.400815 9409 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240501 14:09:27.400877 10241 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240501 14:09:27.400943 8884 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240501 14:09:27.401240 8753 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240501 14:09:27.401216 12000 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240501 14:09:27.401397 11998 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
W20240501 14:09:27.401715 9211 consensus_peers.cc:485] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b39b2a74a9c4dd78fc5541b3ce52772. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:27.401918 11999 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240501 14:09:27.402151 11998 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240501 14:09:27.404000 12000 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.289: config changed from index 105 to 289, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.404376 10241 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 105 to 289, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.404727 8884 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 105 to 289, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.404894 9409 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 105 to 289, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.406677 8612 catalog_manager.cc:5484] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece reported cstate change: config changed from index 105 to 289, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.406704 8753 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 105 to 289, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.459807 11955 raft_consensus.cc:1050] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:27.460204 11955 consensus_queue.cc:237] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 293, Committed index: 293, Last appended: 1.293, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.461542 11576 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240501 14:09:27.461726 10241 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240501 14:09:27.461774 8884 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240501 14:09:27.462191 12103 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240501 14:09:27.462338 8753 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240501 14:09:27.462358 11955 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240501 14:09:27.462455 9144 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240501 14:09:27.462687 12108 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240501 14:09:27.462922 12105 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240501 14:09:27.463150 11955 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240501 14:09:27.464985 11955 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 LEADER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.465476 8753 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.466037 11576 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.466719 9144 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.467232 10241 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.468463 8603 catalog_manager.cc:5484] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 reported cstate change: config changed from index 293 to 294, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" committed_config { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.470407 8884 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.472913 9409 consensus_queue.cc:237] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 294, Committed index: 294, Last appended: 1.294, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.474002 8884 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240501 14:09:27.474522 12103 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240501 14:09:27.475075 9144 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240501 14:09:27.475486 11576 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240501 14:09:27.475615 10241 raft_consensus.cc:1261] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 6b961a167ae64dbc9d102ce5ddf721f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240501 14:09:27.475796 12108 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240501 14:09:27.475951 12107 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240501 14:09:27.476154 12106 consensus_queue.cc:1035] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240501 14:09:27.476550 11923 raft_consensus.cc:1050] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:27.477001 11923 consensus_queue.cc:237] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 350, Committed index: 350, Last appended: 1.350, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.477663 12108 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 LEADER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.478075 11576 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.478809 9014 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240501 14:09:27.479107 8753 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240501 14:09:27.479295 8596 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4b2fe0f030204ecca740b14dfd6605ff with cas_config_opid_index 294: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:27.479430 12000 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240501 14:09:27.479557 8884 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.480274 8612 catalog_manager.cc:5484] T 4b2fe0f030204ecca740b14dfd6605ff P 6b961a167ae64dbc9d102ce5ddf721f3 reported cstate change: config changed from index 294 to 295, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" committed_config { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:27.480795 12000 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240501 14:09:27.481609 11576 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240501 14:09:27.481609 8884 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240501 14:09:27.482197 12000 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240501 14:09:27.482498 11924 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240501 14:09:27.482765 9409 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240501 14:09:27.483433 11924 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240501 14:09:27.484189 8732 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2fe0f030204ecca740b14dfd6605ff with delete_type TABLET_DATA_TOMBSTONED (TS 25737230978048598f426b1a99b27441 not found in new config with opid_index 295) from {username='slave'} at 127.0.0.1:57504
I20240501 14:09:27.484511 12119 tablet_replica.cc:330] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: stopping tablet replica
I20240501 14:09:27.484629 12119 raft_consensus.cc:2229] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:27.484841 12119 raft_consensus.cc:2258] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:27.485523 10241 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.486321 9144 raft_consensus.cc:2932] T 4b2fe0f030204ecca740b14dfd6605ff P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.486878 12119 ts_tablet_manager.cc:1875] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:27.489172 12000 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.351: config changed from index 350 to 351, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.490811 9014 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index 350 to 351, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.490805 8753 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index 350 to 351, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.491614 8607 catalog_manager.cc:5484] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 350 to 351, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:27.492651 9409 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index 350 to 351, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.493031 8884 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index 350 to 351, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.494621 11576 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index 350 to 351, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.497980 12119 ts_tablet_manager.cc:1888] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.294
I20240501 14:09:27.498061 12119 log.cc:1196] T 4b2fe0f030204ecca740b14dfd6605ff P 25737230978048598f426b1a99b27441: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/4b2fe0f030204ecca740b14dfd6605ff
I20240501 14:09:27.498117 9277 consensus_queue.cc:237] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 351, Committed index: 351, Last appended: 1.351, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:27.498579 8597 catalog_manager.cc:4830] TS 25737230978048598f426b1a99b27441 (127.6.153.193:41195): tablet 4b2fe0f030204ecca740b14dfd6605ff (table rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]) successfully deleted
I20240501 14:09:27.499392 8884 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240501 14:09:27.499420 11576 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240501 14:09:27.499645 8753 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240501 14:09:27.499699 11998 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240501 14:09:27.499943 9014 raft_consensus.cc:1261] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240501 14:09:27.500113 11998 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240501 14:09:27.500245 11999 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240501 14:09:27.500777 11738 consensus_queue.cc:1035] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240501 14:09:27.502147 12000 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.352: config changed from index 351 to 352, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.502578 9014 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.502704 8884 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.502909 8753 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.504207 8598 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 188101b7f3784e48975a673952b2bc63 with cas_config_opid_index 351: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:27.504608 11576 raft_consensus.cc:2932] T 188101b7f3784e48975a673952b2bc63 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:27.504827 8609 catalog_manager.cc:5484] T 188101b7f3784e48975a673952b2bc63 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 351 to 352, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:27.507251 9383 tablet_service.cc:1514] Processing DeleteTablet for tablet 188101b7f3784e48975a673952b2bc63 with delete_type TABLET_DATA_TOMBSTONED (TS 6b961a167ae64dbc9d102ce5ddf721f3 not found in new config with opid_index 352) from {username='slave'} at 127.0.0.1:39116
I20240501 14:09:27.507616 12121 tablet_replica.cc:330] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: stopping tablet replica
I20240501 14:09:27.507905 12121 raft_consensus.cc:2229] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:27.508090 12121 raft_consensus.cc:2258] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:27.509307 12121 ts_tablet_manager.cc:1875] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:27.510890 12121 ts_tablet_manager.cc:1888] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.351
I20240501 14:09:27.510953 12121 log.cc:1196] T 188101b7f3784e48975a673952b2bc63 P 6b961a167ae64dbc9d102ce5ddf721f3: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/wals/188101b7f3784e48975a673952b2bc63
I20240501 14:09:27.511385 8596 catalog_manager.cc:4830] TS 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925): tablet 188101b7f3784e48975a673952b2bc63 (table rebalance_test_table_2 [id=bf38945ca7654700b738f68ffcf5f9b8]) successfully deleted
I20240501 14:09:27.676980 9144 consensus_queue.cc:237] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:27.678812 10241 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:27.679260 9409 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:27.679317 9277 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:27.679402 12155 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 14:09:27.679462 9014 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:27.679672 12154 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 14:09:27.680246 11628 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 14:09:27.680567 11628 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 14:09:27.681528 12154 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.681890 10241 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.683846 9409 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.685284 9014 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.685693 8612 catalog_manager.cc:5484] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece reported cstate change: config changed from index -1 to 2, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
W20240501 14:09:27.686691 9080 consensus_peers.cc:485] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab008693c1db49e493c3fac38646c8c0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:27.687038 9277 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:27.753219 12182 ts_tablet_manager.cc:902] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719)
I20240501 14:09:27.753721 12182 tablet_copy_client.cc:308] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:33719
I20240501 14:09:27.754171 9036 tablet_copy_service.cc:140] P 84c7d91c82524b69a9184fcf1949b5b0: Received BeginTabletCopySession request for tablet 2d86cde53ceb4b63842ea9ec09659318 from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:43055)
I20240501 14:09:27.754268 9036 tablet_copy_service.cc:161] P 84c7d91c82524b69a9184fcf1949b5b0: Beginning new tablet copy session on tablet 2d86cde53ceb4b63842ea9ec09659318 from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:43055: session id = 95a4031ef42440ebbdb822d7fb315062-2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:27.755018 9036 tablet_copy_source_session.cc:205] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:27.755825 12182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d86cde53ceb4b63842ea9ec09659318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:27.757586 12182 tablet_copy_client.cc:783] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:27.757712 12182 tablet_copy_client.cc:647] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:27.759228 12182 tablet_copy_client.cc:523] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:27.760185 12182 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:27.836887 12189 ts_tablet_manager.cc:902] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941)
I20240501 14:09:27.837410 12189 tablet_copy_client.cc:308] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:42941
I20240501 14:09:27.837877 9298 tablet_copy_service.cc:140] P 0c0d3d3aae054876a0a522a8325ddd02: Received BeginTabletCopySession request for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:49577)
I20240501 14:09:27.837982 9298 tablet_copy_service.cc:161] P 0c0d3d3aae054876a0a522a8325ddd02: Beginning new tablet copy session on tablet 1b39b2a74a9c4dd78fc5541b3ce52772 from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:49577: session id = 95a4031ef42440ebbdb822d7fb315062-1b39b2a74a9c4dd78fc5541b3ce52772
I20240501 14:09:27.838758 9298 tablet_copy_source_session.cc:205] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:27.839628 12189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b39b2a74a9c4dd78fc5541b3ce52772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:27.841564 12189 tablet_copy_client.cc:783] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:27.841696 12189 tablet_copy_client.cc:647] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:27.843202 12189 tablet_copy_client.cc:523] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:27.844672 12189 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:27.894084 12182 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=244 overwritten=0 applied=244 ignored=0} inserts{seen=868 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:27.894335 12182 tablet_bootstrap.cc:492] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:27.894526 12182 ts_tablet_manager.cc:1367] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.134s user 0.118s sys 0.014s
I20240501 14:09:27.894891 12182 raft_consensus.cc:348] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:27.894992 12182 raft_consensus.cc:729] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:27.895084 12182 consensus_queue.cc:260] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 244, Last appended: 2.244, Last appended by leader: 244, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 244 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:27.895467 12182 ts_tablet_manager.cc:1398] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:09:27.896018 9036 tablet_copy_service.cc:342] P 84c7d91c82524b69a9184fcf1949b5b0: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-2d86cde53ceb4b63842ea9ec09659318 received from {username='slave'} at 127.6.153.200:43055
I20240501 14:09:27.896101 9036 tablet_copy_service.cc:433] P 84c7d91c82524b69a9184fcf1949b5b0: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-2d86cde53ceb4b63842ea9ec09659318 on tablet 2d86cde53ceb4b63842ea9ec09659318 with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:27.993718 12182 ts_tablet_manager.cc:902] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 25737230978048598f426b1a99b27441 (127.6.153.193:41195)
I20240501 14:09:27.994140 12182 tablet_copy_client.cc:308] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:41195
I20240501 14:09:27.997802 8774 tablet_copy_service.cc:140] P 25737230978048598f426b1a99b27441: Received BeginTabletCopySession request for tablet a6b7f72cb79d480a8a39e20c51d922f9 from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:54033)
I20240501 14:09:27.997891 8774 tablet_copy_service.cc:161] P 25737230978048598f426b1a99b27441: Beginning new tablet copy session on tablet a6b7f72cb79d480a8a39e20c51d922f9 from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:54033: session id = 95a4031ef42440ebbdb822d7fb315062-a6b7f72cb79d480a8a39e20c51d922f9
I20240501 14:09:27.998546 8774 tablet_copy_source_session.cc:205] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:27.999284 12182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b7f72cb79d480a8a39e20c51d922f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:28.000849 12182 tablet_copy_client.cc:783] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:28.000993 12182 tablet_copy_client.cc:647] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:28.002600 12182 tablet_copy_client.cc:523] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:28.003650 12182 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:28.019034 12189 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=289 overwritten=0 applied=289 ignored=0} inserts{seen=958 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:28.019281 12189 tablet_bootstrap.cc:492] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:28.019477 12189 ts_tablet_manager.cc:1367] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.175s user 0.167s sys 0.008s
I20240501 14:09:28.019866 12189 raft_consensus.cc:348] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:28.019976 12189 raft_consensus.cc:729] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:28.020061 12189 consensus_queue.cc:260] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 289, Last appended: 1.289, Last appended by leader: 289, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:28.020313 12189 ts_tablet_manager.cc:1398] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:28.020761 9298 tablet_copy_service.cc:342] P 0c0d3d3aae054876a0a522a8325ddd02: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-1b39b2a74a9c4dd78fc5541b3ce52772 received from {username='slave'} at 127.6.153.200:49577
I20240501 14:09:28.020844 9298 tablet_copy_service.cc:433] P 0c0d3d3aae054876a0a522a8325ddd02: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-1b39b2a74a9c4dd78fc5541b3ce52772 on tablet 1b39b2a74a9c4dd78fc5541b3ce52772 with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:28.166203 12182 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=287 overwritten=0 applied=287 ignored=0} inserts{seen=956 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:28.166460 12182 tablet_bootstrap.cc:492] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:28.166662 12182 ts_tablet_manager.cc:1367] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.163s user 0.151s sys 0.007s
I20240501 14:09:28.167042 12182 raft_consensus.cc:348] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:28.167151 12182 raft_consensus.cc:729] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:28.167241 12182 consensus_queue.cc:260] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 287, Last appended: 1.287, Last appended by leader: 287, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 287 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:28.167522 12182 ts_tablet_manager.cc:1398] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:09:28.168154 8774 tablet_copy_service.cc:342] P 25737230978048598f426b1a99b27441: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-a6b7f72cb79d480a8a39e20c51d922f9 received from {username='slave'} at 127.6.153.200:54033
I20240501 14:09:28.168226 8774 tablet_copy_service.cc:433] P 25737230978048598f426b1a99b27441: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-a6b7f72cb79d480a8a39e20c51d922f9 on tablet a6b7f72cb79d480a8a39e20c51d922f9 with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:28.232170 12182 ts_tablet_manager.cc:902] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983)
I20240501 14:09:28.232580 12182 tablet_copy_client.cc:308] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:34983
I20240501 14:09:28.234582 11576 raft_consensus.cc:1203] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.243->[2.244-2.244] Dedup: 2.244->[]
I20240501 14:09:28.236063 9167 tablet_copy_service.cc:140] P 73ea0b7a5e4c45a3814d46e74d62331a: Received BeginTabletCopySession request for tablet ab008693c1db49e493c3fac38646c8c0 from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:53707)
I20240501 14:09:28.236168 9167 tablet_copy_service.cc:161] P 73ea0b7a5e4c45a3814d46e74d62331a: Beginning new tablet copy session on tablet ab008693c1db49e493c3fac38646c8c0 from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:53707: session id = 95a4031ef42440ebbdb822d7fb315062-ab008693c1db49e493c3fac38646c8c0
I20240501 14:09:28.237118 9167 tablet_copy_source_session.cc:205] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:28.237872 12182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab008693c1db49e493c3fac38646c8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:28.239961 12182 tablet_copy_client.cc:783] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:28.240092 12182 tablet_copy_client.cc:647] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:28.241122 12182 tablet_copy_client.cc:523] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:28.242058 12182 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:28.244715 12182 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:28.244882 12182 tablet_bootstrap.cc:492] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:28.245015 12182 ts_tablet_manager.cc:1367] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20240501 14:09:28.245414 12182 raft_consensus.cc:348] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:28.245513 12182 raft_consensus.cc:729] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:28.245601 12182 consensus_queue.cc:260] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:28.245867 12182 ts_tablet_manager.cc:1398] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:28.246313 9167 tablet_copy_service.cc:342] P 73ea0b7a5e4c45a3814d46e74d62331a: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-ab008693c1db49e493c3fac38646c8c0 received from {username='slave'} at 127.6.153.200:53707
I20240501 14:09:28.246390 9167 tablet_copy_service.cc:433] P 73ea0b7a5e4c45a3814d46e74d62331a: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-ab008693c1db49e493c3fac38646c8c0 on tablet ab008693c1db49e493c3fac38646c8c0 with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:28.414532 11576 raft_consensus.cc:1203] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.288->[1.289-1.289] Dedup: 1.289->[]
I20240501 14:09:28.563817 11576 raft_consensus.cc:1203] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.286->[1.287-1.287] Dedup: 1.287->[]
I20240501 14:09:28.616987 11576 raft_consensus.cc:1203] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 14:09:28.755304 12245 raft_consensus.cc:1050] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:28.755729 12245 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 244, Committed index: 244, Last appended: 2.244, Last appended by leader: 159, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 245 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:28.757097 11576 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062 [term 2 LEARNER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 244. Preceding OpId from leader: term: 2 index: 245. (index mismatch)
I20240501 14:09:28.757350 10240 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 244. Preceding OpId from leader: term: 2 index: 245. (index mismatch)
I20240501 14:09:28.757373 9401 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 244. Preceding OpId from leader: term: 2 index: 245. (index mismatch)
I20240501 14:09:28.757535 8749 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 244. Preceding OpId from leader: term: 2 index: 245. (index mismatch)
I20240501 14:09:28.757679 12296 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 245, Last known committed idx: 244, Time since last communication: 0.000s
I20240501 14:09:28.757923 12008 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 245, Last known committed idx: 244, Time since last communication: 0.000s
I20240501 14:09:28.758158 9144 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 244. Preceding OpId from leader: term: 2 index: 245. (index mismatch)
I20240501 14:09:28.758260 12297 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 245, Last known committed idx: 244, Time since last communication: 0.000s
I20240501 14:09:28.758496 12295 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 245, Last known committed idx: 244, Time since last communication: 0.000s
I20240501 14:09:28.758731 12008 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 245, Last known committed idx: 244, Time since last communication: 0.000s
I20240501 14:09:28.760847 12296 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 LEADER]: Committing config change with OpId 2.245: config changed from index 244 to 245, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 245 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.761464 9144 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Committing config change with OpId 2.245: config changed from index 244 to 245, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 245 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.761641 8749 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Committing config change with OpId 2.245: config changed from index 244 to 245, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 245 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.763362 11576 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062 [term 2 FOLLOWER]: Committing config change with OpId 2.245: config changed from index 244 to 245, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 245 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.764308 8607 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 244 to 245, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 245 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.765107 10240 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Committing config change with OpId 2.245: config changed from index 244 to 245, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 245 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.765697 9409 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Committing config change with OpId 2.245: config changed from index 244 to 245, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 245 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.770514 9014 consensus_queue.cc:237] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 245, Committed index: 245, Last appended: 2.245, Last appended by leader: 159, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 246 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:28.771749 9401 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 245. Preceding OpId from leader: term: 2 index: 246. (index mismatch)
I20240501 14:09:28.771826 9144 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 245. Preceding OpId from leader: term: 2 index: 246. (index mismatch)
I20240501 14:09:28.772114 11576 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 245. Preceding OpId from leader: term: 2 index: 246. (index mismatch)
I20240501 14:09:28.772097 12008 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 246, Last known committed idx: 245, Time since last communication: 0.000s
I20240501 14:09:28.772413 8749 raft_consensus.cc:1261] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 245. Preceding OpId from leader: term: 2 index: 246. (index mismatch)
I20240501 14:09:28.772773 12008 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 246, Last known committed idx: 245, Time since last communication: 0.000s
I20240501 14:09:28.772969 12296 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 246, Last known committed idx: 245, Time since last communication: 0.000s
I20240501 14:09:28.773128 12297 consensus_queue.cc:1035] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 246, Last known committed idx: 245, Time since last communication: 0.000s
I20240501 14:09:28.774529 12296 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 LEADER]: Committing config change with OpId 2.246: config changed from index 245 to 246, VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) evicted. New config: { opid_index: 246 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.775246 9144 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Committing config change with OpId 2.246: config changed from index 245 to 246, VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) evicted. New config: { opid_index: 246 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.775274 11576 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 95a4031ef42440ebbdb822d7fb315062 [term 2 FOLLOWER]: Committing config change with OpId 2.246: config changed from index 245 to 246, VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) evicted. New config: { opid_index: 246 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.775528 8749 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Committing config change with OpId 2.246: config changed from index 245 to 246, VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) evicted. New config: { opid_index: 246 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.777388 8609 catalog_manager.cc:5484] T 2d86cde53ceb4b63842ea9ec09659318 P 84c7d91c82524b69a9184fcf1949b5b0 reported cstate change: config changed from index 245 to 246, VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) evicted. New cstate: current_term: 2 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 246 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:28.778074 8599 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2d86cde53ceb4b63842ea9ec09659318 with cas_config_opid_index 245: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:28.778621 9401 raft_consensus.cc:2932] T 2d86cde53ceb4b63842ea9ec09659318 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Committing config change with OpId 2.246: config changed from index 245 to 246, VOTER 9a65600939334f6db8dcca7f07923ece (127.6.153.199) evicted. New config: { opid_index: 246 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.785007 10224 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d86cde53ceb4b63842ea9ec09659318 with delete_type TABLET_DATA_TOMBSTONED (TS 9a65600939334f6db8dcca7f07923ece not found in new config with opid_index 246) from {username='slave'} at 127.0.0.1:35068
I20240501 14:09:28.785439 12310 tablet_replica.cc:330] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: stopping tablet replica
I20240501 14:09:28.785710 12310 raft_consensus.cc:2229] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:28.785935 12310 raft_consensus.cc:2258] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:28.787621 12310 ts_tablet_manager.cc:1875] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:28.789363 12310 ts_tablet_manager.cc:1888] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.245
I20240501 14:09:28.789448 12310 log.cc:1196] T 2d86cde53ceb4b63842ea9ec09659318 P 9a65600939334f6db8dcca7f07923ece: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/wals/2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:28.789952 8597 catalog_manager.cc:4830] TS 9a65600939334f6db8dcca7f07923ece (127.6.153.199:42105): tablet 2d86cde53ceb4b63842ea9ec09659318 (table rebalance_test_table_3 [id=cd32a86b2df240fcb448a8ebaa81f463]) successfully deleted
I20240501 14:09:28.813988 12267 raft_consensus.cc:1050] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:28.814450 12267 consensus_queue.cc:237] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 289, Committed index: 289, Last appended: 1.289, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:28.815907 11576 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240501 14:09:28.816161 8749 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240501 14:09:28.816228 10240 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240501 14:09:28.816408 9409 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240501 14:09:28.816521 8884 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240501 14:09:28.816556 12268 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240501 14:09:28.816797 11738 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240501 14:09:28.816931 12268 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240501 14:09:28.817186 12268 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240501 14:09:28.818142 11738 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240501 14:09:28.820314 12267 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.290: config changed from index 289 to 290, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.820678 8749 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.823150 8609 catalog_manager.cc:5484] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 289 to 290, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:28.823596 8884 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.823947 11576 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.824998 9409 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.828428 10240 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.830338 9278 consensus_queue.cc:237] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 290, Committed index: 290, Last appended: 1.290, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:28.831780 10240 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240501 14:09:28.831900 11576 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240501 14:09:28.832250 12319 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240501 14:09:28.832562 8884 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240501 14:09:28.832655 12268 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240501 14:09:28.832866 9409 raft_consensus.cc:1261] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 0c0d3d3aae054876a0a522a8325ddd02: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240501 14:09:28.832960 12268 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240501 14:09:28.833664 12318 consensus_queue.cc:1035] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240501 14:09:28.835100 12325 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 LEADER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.835538 8884 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.836934 9409 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.837829 8609 catalog_manager.cc:5484] T 1b39b2a74a9c4dd78fc5541b3ce52772 P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: config changed from index 290 to 291, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "0c0d3d3aae054876a0a522a8325ddd02" committed_config { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.838112 8598 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 with cas_config_opid_index 290: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:28.838696 10240 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.838855 11576 raft_consensus.cc:2932] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER 25737230978048598f426b1a99b27441 (127.6.153.193) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.840983 8732 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b39b2a74a9c4dd78fc5541b3ce52772 with delete_type TABLET_DATA_TOMBSTONED (TS 25737230978048598f426b1a99b27441 not found in new config with opid_index 291) from {username='slave'} at 127.0.0.1:57504
I20240501 14:09:28.842703 12336 tablet_replica.cc:330] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: stopping tablet replica
I20240501 14:09:28.842864 12336 raft_consensus.cc:2229] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:28.843103 12336 raft_consensus.cc:2258] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:28.844451 12336 ts_tablet_manager.cc:1875] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:28.846194 12336 ts_tablet_manager.cc:1888] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.290
I20240501 14:09:28.846257 12336 log.cc:1196] T 1b39b2a74a9c4dd78fc5541b3ce52772 P 25737230978048598f426b1a99b27441: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/1b39b2a74a9c4dd78fc5541b3ce52772
I20240501 14:09:28.846669 8597 catalog_manager.cc:4830] TS 25737230978048598f426b1a99b27441 (127.6.153.193:41195): tablet 1b39b2a74a9c4dd78fc5541b3ce52772 (table rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]) successfully deleted
I20240501 14:09:28.942392 12058 raft_consensus.cc:1050] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:28.942828 12058 consensus_queue.cc:237] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 287, Committed index: 287, Last appended: 1.287, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 288 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:28.944319 10240 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 287. Preceding OpId from leader: term: 1 index: 288. (index mismatch)
I20240501 14:09:28.944536 9144 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 287. Preceding OpId from leader: term: 1 index: 288. (index mismatch)
I20240501 14:09:28.944555 9278 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 287. Preceding OpId from leader: term: 1 index: 288. (index mismatch)
I20240501 14:09:28.944555 11576 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 287. Preceding OpId from leader: term: 1 index: 288. (index mismatch)
I20240501 14:09:28.944865 12368 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 288, Last known committed idx: 287, Time since last communication: 0.000s
I20240501 14:09:28.945092 12271 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 288, Last known committed idx: 287, Time since last communication: 0.000s
I20240501 14:09:28.945295 12270 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 288, Last known committed idx: 287, Time since last communication: 0.000s
I20240501 14:09:28.945492 9014 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 287. Preceding OpId from leader: term: 1 index: 288. (index mismatch)
I20240501 14:09:28.945528 12368 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 288, Last known committed idx: 287, Time since last communication: 0.000s
I20240501 14:09:28.946692 12368 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 288, Last known committed idx: 287, Time since last communication: 0.000s
I20240501 14:09:28.947821 12058 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.288: config changed from index 287 to 288, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 288 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.948411 9144 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.288: config changed from index 287 to 288, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 288 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.949534 10240 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.288: config changed from index 287 to 288, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 288 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.949829 11576 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.288: config changed from index 287 to 288, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 288 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.949846 9278 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.288: config changed from index 287 to 288, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 288 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.950963 8612 catalog_manager.cc:5484] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 287 to 288, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 288 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:28.953562 9014 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.288: config changed from index 287 to 288, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 288 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } attrs { replace: true } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.954463 8749 consensus_queue.cc:237] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 288, Committed index: 288, Last appended: 1.288, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:28.956045 10240 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240501 14:09:28.956204 9014 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240501 14:09:28.956394 12374 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240501 14:09:28.956460 9144 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240501 14:09:28.956907 12368 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240501 14:09:28.957037 12271 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240501 14:09:28.957473 11576 raft_consensus.cc:1261] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Refusing update from remote peer 25737230978048598f426b1a99b27441: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240501 14:09:28.957729 12271 consensus_queue.cc:1035] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240501 14:09:28.958662 12271 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 [term 1 LEADER]: Committing config change with OpId 1.289: config changed from index 288 to 289, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.959637 11576 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 288 to 289, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.959825 9144 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 288 to 289, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.960948 8608 catalog_manager.cc:5484] T a6b7f72cb79d480a8a39e20c51d922f9 P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 288 to 289, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New cstate: current_term: 1 leader_uuid: "25737230978048598f426b1a99b27441" committed_config { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:28.961418 8597 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a6b7f72cb79d480a8a39e20c51d922f9 with cas_config_opid_index 288: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:28.961701 10240 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 288 to 289, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.961966 9014 raft_consensus.cc:2932] T a6b7f72cb79d480a8a39e20c51d922f9 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 288 to 289, VOTER 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197) evicted. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } attrs { promote: false } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:28.964035 9248 tablet_service.cc:1514] Processing DeleteTablet for tablet a6b7f72cb79d480a8a39e20c51d922f9 with delete_type TABLET_DATA_TOMBSTONED (TS 0c0d3d3aae054876a0a522a8325ddd02 not found in new config with opid_index 289) from {username='slave'} at 127.0.0.1:54040
I20240501 14:09:28.965943 12376 tablet_replica.cc:330] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: stopping tablet replica
I20240501 14:09:28.966073 12376 raft_consensus.cc:2229] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:28.966208 12376 raft_consensus.cc:2258] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:28.967255 12376 ts_tablet_manager.cc:1875] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:28.969041 12376 ts_tablet_manager.cc:1888] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.288
I20240501 14:09:28.969102 12376 log.cc:1196] T a6b7f72cb79d480a8a39e20c51d922f9 P 0c0d3d3aae054876a0a522a8325ddd02: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/a6b7f72cb79d480a8a39e20c51d922f9
I20240501 14:09:28.969521 8598 catalog_manager.cc:4830] TS 0c0d3d3aae054876a0a522a8325ddd02 (127.6.153.197:42941): tablet a6b7f72cb79d480a8a39e20c51d922f9 (table rebalance_test_table_1 [id=218e82ec93474fe58c6f5feec4b863de]) successfully deleted
I20240501 14:09:29.211360 12269 raft_consensus.cc:1050] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:29.211817 12269 consensus_queue.cc:237] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:29.213519 9278 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:09:29.213572 11576 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:09:29.213680 9409 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:09:29.213685 9014 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:09:29.213928 12393 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 14:09:29.214027 10240 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:09:29.214380 12395 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 14:09:29.214550 12269 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 14:09:29.214830 12269 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 14:09:29.215725 12394 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 14:09:29.216811 12395 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.217161 11576 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.218497 9014 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.219045 10240 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.219213 9278 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.219700 8606 catalog_manager.cc:5484] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 reported cstate change: config changed from index 2 to 3, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.224002 9409 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } attrs { replace: true } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.224289 9144 consensus_queue.cc:237] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:29.225709 10240 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:29.225704 9014 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:29.225932 11576 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:29.225984 9278 raft_consensus.cc:1261] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Refusing update from remote peer 73ea0b7a5e4c45a3814d46e74d62331a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:29.226116 12395 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:29.226289 12269 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:29.226404 12393 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:29.226536 12292 consensus_queue.cc:1035] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:29.228060 12292 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.228506 9278 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.228863 9014 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.229898 8598 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ab008693c1db49e493c3fac38646c8c0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:29.230234 11576 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.231186 8609 catalog_manager.cc:5484] T ab008693c1db49e493c3fac38646c8c0 P 0c0d3d3aae054876a0a522a8325ddd02 reported cstate change: config changed from index 3 to 4, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New cstate: current_term: 1 leader_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.233247 10240 raft_consensus.cc:2932] T ab008693c1db49e493c3fac38646c8c0 P 9a65600939334f6db8dcca7f07923ece [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } } peers { permanent_uuid: "0c0d3d3aae054876a0a522a8325ddd02" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 42941 } } peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "9a65600939334f6db8dcca7f07923ece" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 42105 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:29.233420 9383 tablet_service.cc:1514] Processing DeleteTablet for tablet ab008693c1db49e493c3fac38646c8c0 with delete_type TABLET_DATA_TOMBSTONED (TS 6b961a167ae64dbc9d102ce5ddf721f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39116
I20240501 14:09:29.233736 12401 tablet_replica.cc:330] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: stopping tablet replica
I20240501 14:09:29.233860 12401 raft_consensus.cc:2229] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:29.234014 12401 raft_consensus.cc:2258] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:29.235625 12401 ts_tablet_manager.cc:1875] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:29.237362 12401 ts_tablet_manager.cc:1888] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 14:09:29.237438 12401 log.cc:1196] T ab008693c1db49e493c3fac38646c8c0 P 6b961a167ae64dbc9d102ce5ddf721f3: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/wals/ab008693c1db49e493c3fac38646c8c0
I20240501 14:09:29.237819 8596 catalog_manager.cc:4830] TS 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925): tablet ab008693c1db49e493c3fac38646c8c0 (table TestTable [id=64c83efbafb94a168955877da1bf4dbd]) successfully deleted
I20240501 14:09:29.349296 8996 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:09:29.351753 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:29.352790 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:29.354557 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:29.372752 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:29.376255 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:29.378363 11556 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:29.395772 8863 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:29.431669 9014 consensus_queue.cc:237] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 280, Committed index: 280, Last appended: 1.280, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:29.433380 9144 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 280. Preceding OpId from leader: term: 1 index: 281. (index mismatch)
W20240501 14:09:29.433441 8949 consensus_peers.cc:485] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 -> Peer 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129): Couldn't send request to peer 95a4031ef42440ebbdb822d7fb315062. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f9e94ea8d834320a06db3087864a08c. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:29.433516 8884 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 280. Preceding OpId from leader: term: 1 index: 281. (index mismatch)
I20240501 14:09:29.433843 8749 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 280. Preceding OpId from leader: term: 1 index: 281. (index mismatch)
I20240501 14:09:29.433868 12454 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 281, Last known committed idx: 280, Time since last communication: 0.000s
I20240501 14:09:29.434188 9409 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 280. Preceding OpId from leader: term: 1 index: 281. (index mismatch)
I20240501 14:09:29.434566 12454 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 281, Last known committed idx: 280, Time since last communication: 0.000s
I20240501 14:09:29.434729 12453 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 281, Last known committed idx: 280, Time since last communication: 0.000s
I20240501 14:09:29.434949 12403 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 281, Last known committed idx: 280, Time since last communication: 0.000s
I20240501 14:09:29.437091 12453 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.281: config changed from index -1 to 281, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:29.437389 8884 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.281: config changed from index -1 to 281, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:29.437834 9144 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.281: config changed from index -1 to 281, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:29.441816 9409 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.281: config changed from index -1 to 281, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:29.441869 8609 catalog_manager.cc:5484] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: config changed from index -1 to 281, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:29.441808 8749 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.281: config changed from index -1 to 281, NON_VOTER 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) added. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } } }
I20240501 14:09:30.042614 12475 ts_tablet_manager.cc:902] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: Initiating tablet copy from peer 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719)
I20240501 14:09:30.043063 12475 tablet_copy_client.cc:308] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:33719
I20240501 14:09:30.043535 9036 tablet_copy_service.cc:140] P 84c7d91c82524b69a9184fcf1949b5b0: Received BeginTabletCopySession request for tablet 7f9e94ea8d834320a06db3087864a08c from peer 95a4031ef42440ebbdb822d7fb315062 ({username='slave'} at 127.6.153.200:43055)
I20240501 14:09:30.043634 9036 tablet_copy_service.cc:161] P 84c7d91c82524b69a9184fcf1949b5b0: Beginning new tablet copy session on tablet 7f9e94ea8d834320a06db3087864a08c from peer 95a4031ef42440ebbdb822d7fb315062 at {username='slave'} at 127.6.153.200:43055: session id = 95a4031ef42440ebbdb822d7fb315062-7f9e94ea8d834320a06db3087864a08c
I20240501 14:09:30.044353 9036 tablet_copy_source_session.cc:205] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:30.045202 12475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9e94ea8d834320a06db3087864a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:30.047158 12475 tablet_copy_client.cc:783] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:30.047291 12475 tablet_copy_client.cc:647] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:30.048869 12475 tablet_copy_client.cc:523] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:30.049918 12475 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: Bootstrap starting.
I20240501 14:09:30.212600 12475 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: Bootstrap replayed 1/1 log segments. Stats: ops{read=281 overwritten=0 applied=281 ignored=0} inserts{seen=912 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:30.212890 12475 tablet_bootstrap.cc:492] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: Bootstrap complete.
I20240501 14:09:30.213065 12475 ts_tablet_manager.cc:1367] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: Time spent bootstrapping tablet: real 0.163s user 0.155s sys 0.005s
I20240501 14:09:30.213450 12475 raft_consensus.cc:348] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:30.213563 12475 raft_consensus.cc:729] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95a4031ef42440ebbdb822d7fb315062, State: Initialized, Role: LEARNER
I20240501 14:09:30.213647 12475 consensus_queue.cc:260] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 281, Last appended: 1.281, Last appended by leader: 281, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: true } }
I20240501 14:09:30.214016 12475 ts_tablet_manager.cc:1398] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:30.214498 9036 tablet_copy_service.cc:342] P 84c7d91c82524b69a9184fcf1949b5b0: Request end of tablet copy session 95a4031ef42440ebbdb822d7fb315062-7f9e94ea8d834320a06db3087864a08c received from {username='slave'} at 127.6.153.200:43055
I20240501 14:09:30.214588 9036 tablet_copy_service.cc:433] P 84c7d91c82524b69a9184fcf1949b5b0: ending tablet copy session 95a4031ef42440ebbdb822d7fb315062-7f9e94ea8d834320a06db3087864a08c on tablet 7f9e94ea8d834320a06db3087864a08c with peer 95a4031ef42440ebbdb822d7fb315062
I20240501 14:09:30.582506 11576 raft_consensus.cc:1203] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.280->[1.281-1.281] Dedup: 1.281->[]
I20240501 14:09:31.109726 12518 raft_consensus.cc:1050] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: attempting to promote NON_VOTER 95a4031ef42440ebbdb822d7fb315062 to VOTER
I20240501 14:09:31.110164 12518 consensus_queue.cc:237] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 281, Committed index: 281, Last appended: 1.281, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:31.111850 8749 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 281. Preceding OpId from leader: term: 1 index: 282. (index mismatch)
I20240501 14:09:31.111845 8884 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 281. Preceding OpId from leader: term: 1 index: 282. (index mismatch)
I20240501 14:09:31.111968 9409 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 281. Preceding OpId from leader: term: 1 index: 282. (index mismatch)
I20240501 14:09:31.112416 12521 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 282, Last known committed idx: 281, Time since last communication: 0.000s
I20240501 14:09:31.112480 11576 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 1 LEARNER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 281. Preceding OpId from leader: term: 1 index: 282. (index mismatch)
I20240501 14:09:31.112604 12520 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 282, Last known committed idx: 281, Time since last communication: 0.000s
I20240501 14:09:31.112635 9144 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Refusing update from remote peer 84c7d91c82524b69a9184fcf1949b5b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 281. Preceding OpId from leader: term: 1 index: 282. (index mismatch)
I20240501 14:09:31.112823 12522 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 282, Last known committed idx: 281, Time since last communication: 0.000s
I20240501 14:09:31.113301 12521 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 282, Last known committed idx: 281, Time since last communication: 0.000s
I20240501 14:09:31.113443 12518 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 282, Last known committed idx: 281, Time since last communication: 0.000s
I20240501 14:09:31.115545 12522 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Committing config change with OpId 1.282: config changed from index 281 to 282, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:31.115949 11576 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Committing config change with OpId 1.282: config changed from index 281 to 282, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:31.116081 8749 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Committing config change with OpId 1.282: config changed from index 281 to 282, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:31.118400 8603 catalog_manager.cc:5484] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 reported cstate change: config changed from index 281 to 282, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84c7d91c82524b69a9184fcf1949b5b0" committed_config { opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:31.119252 8884 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Committing config change with OpId 1.282: config changed from index 281 to 282, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:31.119391 9409 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Committing config change with OpId 1.282: config changed from index 281 to 282, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:31.122001 9144 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Committing config change with OpId 1.282: config changed from index 281 to 282, 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:31.307132 9014 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c"
dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49402
I20240501 14:09:31.307354 9014 raft_consensus.cc:595] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:31.509547 12529 raft_consensus.cc:979] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: : Instructing follower f2e6587c4d054aa696aaa7ff76880a2f to start an election
I20240501 14:09:31.509719 12529 raft_consensus.cc:1067] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Signalling peer f2e6587c4d054aa696aaa7ff76880a2f to start an election
I20240501 14:09:31.510300 8884 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c"
dest_uuid: "f2e6587c4d054aa696aaa7ff76880a2f"
from {username='slave'} at 127.6.153.195:45825
I20240501 14:09:31.510473 8884 raft_consensus.cc:482] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:31.510547 8884 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:31.511680 8884 raft_consensus.cc:504] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:31.512356 8884 leader_election.cc:288] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 2 election: Requested vote from peers 73ea0b7a5e4c45a3814d46e74d62331a (127.6.153.196:34983), 6b961a167ae64dbc9d102ce5ddf721f3 (127.6.153.198:32925), 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719), 25737230978048598f426b1a99b27441 (127.6.153.193:41195), 95a4031ef42440ebbdb822d7fb315062 (127.6.153.200:39129)
I20240501 14:09:31.512624 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 282 } } ignore_live_leader: true dest_uuid: "6b961a167ae64dbc9d102ce5ddf721f3"
I20240501 14:09:31.512631 9144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 282 } } ignore_live_leader: true dest_uuid: "73ea0b7a5e4c45a3814d46e74d62331a"
I20240501 14:09:31.512754 9409 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:31.512754 9144 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:31.512878 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 282 } } ignore_live_leader: true dest_uuid: "84c7d91c82524b69a9184fcf1949b5b0"
I20240501 14:09:31.512991 9014 raft_consensus.cc:3032] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:31.512977 8749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 282 } } ignore_live_leader: true dest_uuid: "25737230978048598f426b1a99b27441"
I20240501 14:09:31.513027 9014 raft_consensus.cc:729] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84c7d91c82524b69a9184fcf1949b5b0, State: Running, Role: LEADER
I20240501 14:09:31.513082 8749 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:31.513176 9014 consensus_queue.cc:260] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 282, Committed index: 282, Last appended: 1.282, Last appended by leader: 282, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:31.513386 9014 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:31.513721 9144 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 2.
I20240501 14:09:31.513721 9409 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 2.
I20240501 14:09:31.514077 8749 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 2.
I20240501 14:09:31.514283 9014 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 2.
I20240501 14:09:31.514505 8819 leader_election.cc:302] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 6 voters: 4 yes votes; 0 no votes. yes voters: 25737230978048598f426b1a99b27441, 6b961a167ae64dbc9d102ce5ddf721f3, 73ea0b7a5e4c45a3814d46e74d62331a, f2e6587c4d054aa696aaa7ff76880a2f; no voters:
I20240501 14:09:31.514673 11786 raft_consensus.cc:2781] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:31.514811 11786 raft_consensus.cc:686] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 2 LEADER]: Becoming Leader. State: Replica: f2e6587c4d054aa696aaa7ff76880a2f, State: Running, Role: LEADER
I20240501 14:09:31.514791 11576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9e94ea8d834320a06db3087864a08c" candidate_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 282 } } ignore_live_leader: true dest_uuid: "95a4031ef42440ebbdb822d7fb315062"
I20240501 14:09:31.514957 11786 consensus_queue.cc:237] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 282, Committed index: 282, Last appended: 1.282, Last appended by leader: 282, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:31.514992 11576 raft_consensus.cc:3037] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:31.516055 11576 raft_consensus.cc:2454] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2e6587c4d054aa696aaa7ff76880a2f in term 2.
I20240501 14:09:31.517474 8612 catalog_manager.cc:5484] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f reported cstate change: term changed from 1 to 2, leader changed from 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) to f2e6587c4d054aa696aaa7ff76880a2f (127.6.153.194). New cstate: current_term: 2 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: 282 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:31.925638 9144 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 282. Preceding OpId from leader: term: 2 index: 283. (index mismatch)
I20240501 14:09:31.926026 12551 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 283, Last known committed idx: 282, Time since last communication: 0.000s
I20240501 14:09:32.070238 8749 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 282. Preceding OpId from leader: term: 2 index: 283. (index mismatch)
I20240501 14:09:32.070611 12551 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 283, Last known committed idx: 282, Time since last communication: 0.000s
I20240501 14:09:32.100428 9409 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 282. Preceding OpId from leader: term: 2 index: 283. (index mismatch)
I20240501 14:09:32.100783 12551 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 283, Last known committed idx: 282, Time since last communication: 0.000s
I20240501 14:09:32.103638 9014 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 282. Preceding OpId from leader: term: 2 index: 283. (index mismatch)
I20240501 14:09:32.104300 12564 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c7d91c82524b69a9184fcf1949b5b0" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 33719 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 283, Last known committed idx: 282, Time since last communication: 0.000s
I20240501 14:09:32.106194 11576 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 282. Preceding OpId from leader: term: 2 index: 283. (index mismatch)
I20240501 14:09:32.106633 12569 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 283, Last known committed idx: 282, Time since last communication: 0.000s
I20240501 14:09:32.685747 8884 consensus_queue.cc:237] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 283, Committed index: 283, Last appended: 2.283, Last appended by leader: 282, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 284 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } }
I20240501 14:09:32.687100 11576 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 283. Preceding OpId from leader: term: 2 index: 284. (index mismatch)
I20240501 14:09:32.687100 9409 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 283. Preceding OpId from leader: term: 2 index: 284. (index mismatch)
I20240501 14:09:32.687165 8749 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 283. Preceding OpId from leader: term: 2 index: 284. (index mismatch)
I20240501 14:09:32.687275 9144 raft_consensus.cc:1261] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Refusing update from remote peer f2e6587c4d054aa696aaa7ff76880a2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 283. Preceding OpId from leader: term: 2 index: 284. (index mismatch)
I20240501 14:09:32.687697 12568 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 284, Last known committed idx: 283, Time since last communication: 0.000s
I20240501 14:09:32.687892 12569 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 284, Last known committed idx: 283, Time since last communication: 0.000s
I20240501 14:09:32.688051 12564 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 284, Last known committed idx: 283, Time since last communication: 0.000s
I20240501 14:09:32.688233 12592 consensus_queue.cc:1035] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 284, Last known committed idx: 283, Time since last communication: 0.000s
I20240501 14:09:32.689993 12568 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P f2e6587c4d054aa696aaa7ff76880a2f [term 2 LEADER]: Committing config change with OpId 2.284: config changed from index 282 to 284, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 284 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:32.690940 8749 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 [term 2 FOLLOWER]: Committing config change with OpId 2.284: config changed from index 282 to 284, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 284 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:32.691082 9144 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 73ea0b7a5e4c45a3814d46e74d62331a [term 2 FOLLOWER]: Committing config change with OpId 2.284: config changed from index 282 to 284, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 284 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:32.691994 11576 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 95a4031ef42440ebbdb822d7fb315062 [term 2 FOLLOWER]: Committing config change with OpId 2.284: config changed from index 282 to 284, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 284 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:32.692346 9409 raft_consensus.cc:2932] T 7f9e94ea8d834320a06db3087864a08c P 6b961a167ae64dbc9d102ce5ddf721f3 [term 2 FOLLOWER]: Committing config change with OpId 2.284: config changed from index 282 to 284, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New config: { opid_index: 284 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:32.693630 8609 catalog_manager.cc:5484] T 7f9e94ea8d834320a06db3087864a08c P 25737230978048598f426b1a99b27441 reported cstate change: config changed from index 282 to 284, VOTER 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195) evicted. New cstate: current_term: 2 leader_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" committed_config { opid_index: 284 OBSOLETE_local: false peers { permanent_uuid: "73ea0b7a5e4c45a3814d46e74d62331a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 34983 } } peers { permanent_uuid: "6b961a167ae64dbc9d102ce5ddf721f3" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 32925 } } peers { permanent_uuid: "25737230978048598f426b1a99b27441" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 41195 } } peers { permanent_uuid: "f2e6587c4d054aa696aaa7ff76880a2f" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 35221 } } peers { permanent_uuid: "95a4031ef42440ebbdb822d7fb315062" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 39129 } attrs { promote: false } } }
I20240501 14:09:32.694052 8596 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7f9e94ea8d834320a06db3087864a08c with cas_config_opid_index 282: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:32.696813 8996 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f9e94ea8d834320a06db3087864a08c with delete_type TABLET_DATA_TOMBSTONED (TS 84c7d91c82524b69a9184fcf1949b5b0 not found in new config with opid_index 284) from {username='slave'} at 127.0.0.1:34080
I20240501 14:09:32.697332 12598 tablet_replica.cc:330] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: stopping tablet replica
I20240501 14:09:32.697567 12598 raft_consensus.cc:2229] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:32.697754 12598 raft_consensus.cc:2258] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:32.699096 12598 ts_tablet_manager.cc:1875] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:32.700886 12598 ts_tablet_manager.cc:1888] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.283
I20240501 14:09:32.700949 12598 log.cc:1196] T 7f9e94ea8d834320a06db3087864a08c P 84c7d91c82524b69a9184fcf1949b5b0: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/7f9e94ea8d834320a06db3087864a08c
I20240501 14:09:32.701447 8599 catalog_manager.cc:4830] TS 84c7d91c82524b69a9184fcf1949b5b0 (127.6.153.195:33719): tablet 7f9e94ea8d834320a06db3087864a08c (table rebalance_test_table_0 [id=091b87e33e224e50b05a4f016867ef06]) successfully deleted
I20240501 14:09:32.813463 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:32.816244 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:32.817070 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:32.821398 8996 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:32.836428 8863 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:32.840400 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:32.841845 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:32.842486 11556 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:32.930351 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:32.932760 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:32.938881 8996 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:32.940558 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:32.947865 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:32.964056 11556 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:32.964188 8863 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:32.970655 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:33.059131 8732 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:33.059607 9383 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:33.059827 9248 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:09:33.067484 8996 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:09:33.077529 9126 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:09:33.077529 10224 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:33.077735 11556 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:33.090762 8863 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
9fa87eb4295445af9eb74cc9bbb89994 | 127.6.153.254:42171 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.6.153.212:42933 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
0c0d3d3aae054876a0a522a8325ddd02 | 127.6.153.197:42941 | HEALTHY | <none> | 3 | 0
25737230978048598f426b1a99b27441 | 127.6.153.193:41195 | HEALTHY | <none> | 3 | 0
6b961a167ae64dbc9d102ce5ddf721f3 | 127.6.153.198:32925 | HEALTHY | <none> | 1 | 0
73ea0b7a5e4c45a3814d46e74d62331a | 127.6.153.196:34983 | HEALTHY | <none> | 2 | 0
84c7d91c82524b69a9184fcf1949b5b0 | 127.6.153.195:33719 | HEALTHY | <none> | 4 | 0
95a4031ef42440ebbdb822d7fb315062 | 127.6.153.200:39129 | HEALTHY | <none> | 0 | 0
9a65600939334f6db8dcca7f07923ece | 127.6.153.199:42105 | HEALTHY | <none> | 0 | 0
f2e6587c4d054aa696aaa7ff76880a2f | 127.6.153.194:35221 | HEALTHY | <none> | 3 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.6.153.193 | experimental | 127.6.153.193:41195
local_ip_for_outbound_sockets | 127.6.153.194 | experimental | 127.6.153.194:35221
local_ip_for_outbound_sockets | 127.6.153.195 | experimental | 127.6.153.195:33719
local_ip_for_outbound_sockets | 127.6.153.196 | experimental | 127.6.153.196:34983
local_ip_for_outbound_sockets | 127.6.153.197 | experimental | 127.6.153.197:42941
local_ip_for_outbound_sockets | 127.6.153.198 | experimental | 127.6.153.198:32925
local_ip_for_outbound_sockets | 127.6.153.199 | experimental | 127.6.153.199:42105
local_ip_for_outbound_sockets | 127.6.153.200 | experimental | 127.6.153.200:39129
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.6.153.193:41195
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.153.194:35221
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.153.195:33719
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.153.196:34983
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.153.197:42941
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.153.198:32925
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.6.153.199:42105
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.6.153.200:39129
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.6.153.212:42933 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 14:09:33.119410 6759 log_verifier.cc:126] Checking tablet 188101b7f3784e48975a673952b2bc63
I20240501 14:09:33.262396 6759 log_verifier.cc:177] Verified matching terms for 352 ops in tablet 188101b7f3784e48975a673952b2bc63
I20240501 14:09:33.262616 6759 log_verifier.cc:126] Checking tablet 1b39b2a74a9c4dd78fc5541b3ce52772
I20240501 14:09:33.391857 6759 log_verifier.cc:177] Verified matching terms for 291 ops in tablet 1b39b2a74a9c4dd78fc5541b3ce52772
I20240501 14:09:33.392051 6759 log_verifier.cc:126] Checking tablet 203c4d95b91f495f91f96b9308d1efc3
I20240501 14:09:33.506768 6759 log_verifier.cc:177] Verified matching terms for 255 ops in tablet 203c4d95b91f495f91f96b9308d1efc3
I20240501 14:09:33.506989 6759 log_verifier.cc:126] Checking tablet 2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:33.619050 6759 log_verifier.cc:177] Verified matching terms for 246 ops in tablet 2d86cde53ceb4b63842ea9ec09659318
I20240501 14:09:33.619244 6759 log_verifier.cc:126] Checking tablet 30ef391ceec44fd4b7195b41d7732843
I20240501 14:09:33.747092 6759 log_verifier.cc:177] Verified matching terms for 293 ops in tablet 30ef391ceec44fd4b7195b41d7732843
I20240501 14:09:33.747303 6759 log_verifier.cc:126] Checking tablet 4323f538490b4bf99fa7f96f80c8a88f
I20240501 14:09:33.863291 6759 log_verifier.cc:177] Verified matching terms for 279 ops in tablet 4323f538490b4bf99fa7f96f80c8a88f
I20240501 14:09:33.863488 6759 log_verifier.cc:126] Checking tablet 4b2fe0f030204ecca740b14dfd6605ff
I20240501 14:09:33.988749 6759 log_verifier.cc:177] Verified matching terms for 295 ops in tablet 4b2fe0f030204ecca740b14dfd6605ff
I20240501 14:09:33.988932 6759 log_verifier.cc:126] Checking tablet 560c845cf74c4df1bf33732abc84620d
I20240501 14:09:34.109090 6759 log_verifier.cc:177] Verified matching terms for 293 ops in tablet 560c845cf74c4df1bf33732abc84620d
I20240501 14:09:34.109258 6759 log_verifier.cc:126] Checking tablet 7eac39a2ecdc40f0929ab332508189cd
I20240501 14:09:34.222807 6759 log_verifier.cc:177] Verified matching terms for 254 ops in tablet 7eac39a2ecdc40f0929ab332508189cd
I20240501 14:09:34.222985 6759 log_verifier.cc:126] Checking tablet 7f9e94ea8d834320a06db3087864a08c
I20240501 14:09:34.344874 6759 log_verifier.cc:177] Verified matching terms for 284 ops in tablet 7f9e94ea8d834320a06db3087864a08c
I20240501 14:09:34.345091 6759 log_verifier.cc:126] Checking tablet 858bf448fd5249f0a86d17d6d011a6b4
I20240501 14:09:34.468845 6759 log_verifier.cc:177] Verified matching terms for 282 ops in tablet 858bf448fd5249f0a86d17d6d011a6b4
I20240501 14:09:34.469015 6759 log_verifier.cc:126] Checking tablet 96ab0c0bda54481b95b0fdf9d8da84df
I20240501 14:09:34.596319 6759 log_verifier.cc:177] Verified matching terms for 301 ops in tablet 96ab0c0bda54481b95b0fdf9d8da84df
I20240501 14:09:34.596501 6759 log_verifier.cc:126] Checking tablet a6b7f72cb79d480a8a39e20c51d922f9
I20240501 14:09:34.725630 6759 log_verifier.cc:177] Verified matching terms for 289 ops in tablet a6b7f72cb79d480a8a39e20c51d922f9
I20240501 14:09:34.725807 6759 log_verifier.cc:126] Checking tablet ab008693c1db49e493c3fac38646c8c0
I20240501 14:09:34.750452 6759 log_verifier.cc:177] Verified matching terms for 4 ops in tablet ab008693c1db49e493c3fac38646c8c0
I20240501 14:09:34.750514 6759 log_verifier.cc:126] Checking tablet d21fe5dd65fe4fbea85fecc8ce19bc43
I20240501 14:09:34.895931 6759 log_verifier.cc:177] Verified matching terms for 377 ops in tablet d21fe5dd65fe4fbea85fecc8ce19bc43
I20240501 14:09:34.896198 6759 log_verifier.cc:126] Checking tablet e72446fcffed439c8653937330e5b8ee
I20240501 14:09:35.034963 6759 log_verifier.cc:177] Verified matching terms for 348 ops in tablet e72446fcffed439c8653937330e5b8ee
I20240501 14:09:35.036449 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 8672
I20240501 14:09:35.043457 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 8803
I20240501 14:09:35.049572 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 8934
I20240501 14:09:35.055151 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 9065
I20240501 14:09:35.060637 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 9196
I20240501 14:09:35.065634 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 9327
I20240501 14:09:35.070762 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 10161
I20240501 14:09:35.075264 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 11493
I20240501 14:09:35.079748 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 8581
2024-05-01T14:09:35Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/7 (39183 ms)
[----------] 1 test from RebalanceParamTest (39183 ms total)
[----------] 1 test from TserverAddedDuringRebalancingTest
[ RUN ] TserverAddedDuringRebalancingTest.TserverStarts/1
I20240501 14:09:35.159065 6759 test_util.cc:268] Using random seed: 155759378
I20240501 14:09:35.159929 6759 ts_itest-base.cc:112] Starting cluster with:
I20240501 14:09:35.159977 6759 ts_itest-base.cc:113] --------------
I20240501 14:09:35.159992 6759 ts_itest-base.cc:114] 8 tablet servers
I20240501 14:09:35.160004 6759 ts_itest-base.cc:115] 3 replicas per TS
I20240501 14:09:35.160018 6759 ts_itest-base.cc:116] --------------
2024-05-01T14:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T14:09:35Z Disabled control of system clock
I20240501 14:09:35.169342 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:41367
--webserver_interface=127.6.153.254
--webserver_port=0
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.153.254:41367
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 14:09:35.268483 12723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:35.268678 12723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:35.268725 12723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:35.271674 12723 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:35.271732 12723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 14:09:35.271756 12723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:35.271788 12723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 14:09:35.271816 12723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 14:09:35.275305 12723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.6.153.254:41367
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:41367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.153.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.12723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:35.276108 12723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:35.277110 12723 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:35.284080 12728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:35.284406 12729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:35.284778 12731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:35.285272 12723 server_base.cc:1034] running on GCE node
I20240501 14:09:35.285708 12723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:35.286491 12723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:35.287626 12723 hybrid_clock.cc:648] HybridClock initialized: now 1714572575287617 us; error 27 us; skew 500 ppm
I20240501 14:09:35.289278 12723 webserver.cc:460] Webserver started at http://127.6.153.254:40751/ using document root <none> and password file <none>
I20240501 14:09:35.289763 12723 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:35.289817 12723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:35.289975 12723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:35.291504 12723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "e1cb11679f4c4a2f9311a613a170b3d7"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.291924 12723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "e1cb11679f4c4a2f9311a613a170b3d7"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.294903 12723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:09:35.296856 12737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.297746 12723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:35.297852 12723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e1cb11679f4c4a2f9311a613a170b3d7"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.297928 12723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:35.309485 12723 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:35.309969 12723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:35.310103 12723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:35.316108 12723 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.254:41367
I20240501 14:09:35.316121 12790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.254:41367 every 8 connection(s)
I20240501 14:09:35.316962 12723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 14:09:35.319205 12791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:35.323272 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 12723
I20240501 14:09:35.323403 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 14:09:35.323755 12791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7: Bootstrap starting.
I20240501 14:09:35.326113 12791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:35.326892 12791 log.cc:824] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:35.328594 12791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7: No bootstrap required, opened a new log
I20240501 14:09:35.331480 12791 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cb11679f4c4a2f9311a613a170b3d7" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 41367 } }
I20240501 14:09:35.331686 12791 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:35.331743 12791 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1cb11679f4c4a2f9311a613a170b3d7, State: Initialized, Role: FOLLOWER
I20240501 14:09:35.332167 12791 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cb11679f4c4a2f9311a613a170b3d7" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 41367 } }
I20240501 14:09:35.332314 12791 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:09:35.332377 12791 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:09:35.332480 12791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:35.333154 12791 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cb11679f4c4a2f9311a613a170b3d7" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 41367 } }
I20240501 14:09:35.333448 12791 leader_election.cc:302] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1cb11679f4c4a2f9311a613a170b3d7; no voters:
I20240501 14:09:35.333681 12791 leader_election.cc:288] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:09:35.333760 12796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:35.333951 12796 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [term 1 LEADER]: Becoming Leader. State: Replica: e1cb11679f4c4a2f9311a613a170b3d7, State: Running, Role: LEADER
I20240501 14:09:35.334244 12796 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cb11679f4c4a2f9311a613a170b3d7" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 41367 } }
I20240501 14:09:35.334604 12791 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 14:09:35.336061 12798 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1cb11679f4c4a2f9311a613a170b3d7. Latest consensus state: current_term: 1 leader_uuid: "e1cb11679f4c4a2f9311a613a170b3d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cb11679f4c4a2f9311a613a170b3d7" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 41367 } } }
I20240501 14:09:35.336088 12797 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1cb11679f4c4a2f9311a613a170b3d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cb11679f4c4a2f9311a613a170b3d7" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 41367 } } }
I20240501 14:09:35.336252 12798 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [sys.catalog]: This master's current role is: LEADER
I20240501 14:09:35.336341 12797 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7 [sys.catalog]: This master's current role is: LEADER
I20240501 14:09:35.336756 12808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 14:09:35.339190 12808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 14:09:35.343660 12808 catalog_manager.cc:1347] Generated new cluster ID: 4727aafeb7d5472dbf22c95e56291a8e
I20240501 14:09:35.343724 12808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 14:09:35.358935 12808 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 14:09:35.359793 12808 catalog_manager.cc:1504] Loading token signing keys...
I20240501 14:09:35.365495 12808 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e1cb11679f4c4a2f9311a613a170b3d7: Generated new TSK 0
I20240501 14:09:35.366055 12808 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 14:09:35.368461 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.193:0
--local_ip_for_outbound_sockets=127.6.153.193
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 14:09:35.473088 12815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:35.473274 12815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:35.473332 12815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:35.476220 12815 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:35.476285 12815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:35.476364 12815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.193
I20240501 14:09:35.479780 12815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.153.193
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.12815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.193
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:35.480509 12815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:35.481489 12815 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:35.489435 12820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:35.489822 12821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:35.490128 12823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:35.490538 12815 server_base.cc:1034] running on GCE node
I20240501 14:09:35.490860 12815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:35.491351 12815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:35.492501 12815 hybrid_clock.cc:648] HybridClock initialized: now 1714572575492483 us; error 34 us; skew 500 ppm
I20240501 14:09:35.494144 12815 webserver.cc:460] Webserver started at http://127.6.153.193:38417/ using document root <none> and password file <none>
I20240501 14:09:35.494623 12815 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:35.494678 12815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:35.494834 12815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:35.496342 12815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.496773 12815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.499716 12815 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:09:35.501703 12829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.502588 12815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:35.502688 12815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.502761 12815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:35.523769 12815 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:35.524291 12815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:35.524428 12815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:35.524904 12815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:35.525692 12815 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:35.525738 12815 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.525794 12815 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:35.525820 12815 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.533383 12815 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.193:45393
I20240501 14:09:35.533422 12943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.193:45393 every 8 connection(s)
I20240501 14:09:35.534188 12815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 14:09:35.541973 12944 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:35.542145 12944 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:35.542606 12944 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:35.543795 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 12815
I20240501 14:09:35.543891 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 14:09:35.543989 12751 ts_manager.cc:194] Registered new tserver with Master: a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:35.545171 12751 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.193:38305
I20240501 14:09:35.546056 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.194:0
--local_ip_for_outbound_sockets=127.6.153.194
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 14:09:35.646309 12947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:35.646515 12947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:35.646581 12947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:35.650439 12947 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:35.650501 12947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:35.650592 12947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.194
I20240501 14:09:35.654724 12947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.153.194
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.12947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.194
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:35.655550 12947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:35.656564 12947 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:35.664364 12955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:35.664767 12952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:35.665164 12953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:35.665545 12947 server_base.cc:1034] running on GCE node
I20240501 14:09:35.665882 12947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:35.666361 12947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:35.667490 12947 hybrid_clock.cc:648] HybridClock initialized: now 1714572575667474 us; error 30 us; skew 500 ppm
I20240501 14:09:35.669198 12947 webserver.cc:460] Webserver started at http://127.6.153.194:41807/ using document root <none> and password file <none>
I20240501 14:09:35.669690 12947 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:35.669740 12947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:35.669909 12947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:35.671504 12947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "3021737e50984ca4bb2ad151dc4776ff"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.671948 12947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "3021737e50984ca4bb2ad151dc4776ff"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.674935 12947 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:35.676937 12961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.677907 12947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:35.678007 12947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "3021737e50984ca4bb2ad151dc4776ff"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.678081 12947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:35.696419 12947 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:35.696936 12947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:35.697075 12947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:35.697536 12947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:35.698349 12947 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:35.698393 12947 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.698443 12947 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:35.698472 12947 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.706781 12947 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.194:36421
I20240501 14:09:35.706828 13074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.194:36421 every 8 connection(s)
I20240501 14:09:35.707644 12947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 14:09:35.710997 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 12947
I20240501 14:09:35.711084 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 14:09:35.713351 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:0
--local_ip_for_outbound_sockets=127.6.153.195
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:35.716274 13075 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:35.716480 13075 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:35.716948 13075 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:35.717801 12751 ts_manager.cc:194] Registered new tserver with Master: 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:35.718331 12751 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.194:54497
W20240501 14:09:35.813598 13078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:35.813786 13078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:35.813846 13078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:35.816820 13078 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:35.816881 13078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:35.816964 13078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:09:35.820370 13078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.13078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:35.821100 13078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:35.822100 13078 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:35.829855 13087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:35.830175 13084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:35.830555 13085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:35.831046 13078 server_base.cc:1034] running on GCE node
I20240501 14:09:35.831435 13078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:35.831935 13078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:35.833081 13078 hybrid_clock.cc:648] HybridClock initialized: now 1714572575833063 us; error 30 us; skew 500 ppm
I20240501 14:09:35.834789 13078 webserver.cc:460] Webserver started at http://127.6.153.195:42359/ using document root <none> and password file <none>
I20240501 14:09:35.835299 13078 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:35.835381 13078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:35.835546 13078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:35.837186 13078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "b6ea309b0a4a44dc9415766897152739"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.837627 13078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "b6ea309b0a4a44dc9415766897152739"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.840679 13078 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20240501 14:09:35.842674 13093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.843611 13078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:35.843699 13078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "b6ea309b0a4a44dc9415766897152739"
format_stamp: "Formatted at 2024-05-01 14:09:35 on dist-test-slave-cmf0"
I20240501 14:09:35.843773 13078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:35.891039 13078 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:35.891597 13078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:35.891729 13078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:35.892194 13078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:35.892989 13078 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:35.893033 13078 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.893074 13078 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:35.893101 13078 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:35.900511 13078 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:38667
I20240501 14:09:35.900557 13206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:38667 every 8 connection(s)
I20240501 14:09:35.901331 13078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:09:35.908809 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 13078
I20240501 14:09:35.908911 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 14:09:35.909165 13207 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:35.909379 13207 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:35.909847 13207 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:35.910737 12751 ts_manager.cc:194] Registered new tserver with Master: b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:35.910914 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.196:0
--local_ip_for_outbound_sockets=127.6.153.196
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:35.911402 12751 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:36035
W20240501 14:09:36.008778 13210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:36.008966 13210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:36.009025 13210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:36.011945 13210 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:36.012003 13210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:36.012082 13210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.196
I20240501 14:09:36.015517 13210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.153.196
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.13210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.196
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:36.016315 13210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:36.017329 13210 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:36.024940 13216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.025205 13218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.025575 13215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:36.026048 13210 server_base.cc:1034] running on GCE node
I20240501 14:09:36.026387 13210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:36.026922 13210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:36.028079 13210 hybrid_clock.cc:648] HybridClock initialized: now 1714572576028057 us; error 32 us; skew 500 ppm
I20240501 14:09:36.029675 13210 webserver.cc:460] Webserver started at http://127.6.153.196:46699/ using document root <none> and password file <none>
I20240501 14:09:36.030130 13210 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:36.030179 13210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:36.030328 13210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:36.031848 13210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "879199fbf918416cb23b0376dde0aa6e"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.032248 13210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "879199fbf918416cb23b0376dde0aa6e"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.035279 13210 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:09:36.037262 13224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.038128 13210 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:36.038219 13210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "879199fbf918416cb23b0376dde0aa6e"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.038302 13210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:36.051800 13210 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:36.052237 13210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:36.052371 13210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:36.052798 13210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:36.053545 13210 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:36.053589 13210 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.053638 13210 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:36.053666 13210 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.060997 13210 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.196:33309
I20240501 14:09:36.061048 13339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.196:33309 every 8 connection(s)
I20240501 14:09:36.061796 13210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 14:09:36.065547 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 13210
I20240501 14:09:36.065647 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 14:09:36.067736 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.197:0
--local_ip_for_outbound_sockets=127.6.153.197
--webserver_interface=127.6.153.197
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:36.070009 13340 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:36.070214 13340 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:36.070751 13340 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:36.071730 12751 ts_manager.cc:194] Registered new tserver with Master: 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:36.072258 12751 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.196:40995
W20240501 14:09:36.168954 13343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:36.169126 13343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:36.169173 13343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:36.172060 13343 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:36.172111 13343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:36.172194 13343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.197
I20240501 14:09:36.175596 13343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.6.153.197
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.13343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.197
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:36.176318 13343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:36.177306 13343 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:36.184859 13351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.185180 13349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.185528 13348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:36.186007 13343 server_base.cc:1034] running on GCE node
I20240501 14:09:36.186340 13343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:36.186829 13343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:36.187970 13343 hybrid_clock.cc:648] HybridClock initialized: now 1714572576187960 us; error 24 us; skew 500 ppm
I20240501 14:09:36.189664 13343 webserver.cc:460] Webserver started at http://127.6.153.197:39881/ using document root <none> and password file <none>
I20240501 14:09:36.190160 13343 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:36.190224 13343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:36.190399 13343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:36.191946 13343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "2986bfcc5da34cbca2cad1643b6e35c9"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.192447 13343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "2986bfcc5da34cbca2cad1643b6e35c9"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.195565 13343 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:09:36.197562 13357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.198479 13343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:36.198585 13343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "2986bfcc5da34cbca2cad1643b6e35c9"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.198659 13343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:36.223013 13343 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:36.223572 13343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:36.223706 13343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:36.224165 13343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:36.224988 13343 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:36.225034 13343 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.225076 13343 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:36.225104 13343 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.232452 13343 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.197:37879
I20240501 14:09:36.232483 13471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.197:37879 every 8 connection(s)
I20240501 14:09:36.233265 13343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 14:09:36.241147 13472 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:36.241322 13472 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:36.241746 13472 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:36.242617 12751 ts_manager.cc:194] Registered new tserver with Master: 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879)
I20240501 14:09:36.243175 12751 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.197:59913
I20240501 14:09:36.243242 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 13343
I20240501 14:09:36.243363 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 14:09:36.245270 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.198:0
--local_ip_for_outbound_sockets=127.6.153.198
--webserver_interface=127.6.153.198
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 14:09:36.344978 13475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:36.345144 13475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:36.345185 13475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:36.348104 13475 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:36.348157 13475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:36.348225 13475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.198
I20240501 14:09:36.351670 13475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.198:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.6.153.198
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.13475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.198
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:36.352389 13475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:36.353379 13475 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:36.360875 13480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.361172 13481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.361606 13483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:36.361966 13475 server_base.cc:1034] running on GCE node
I20240501 14:09:36.362284 13475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:36.362756 13475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:36.363898 13475 hybrid_clock.cc:648] HybridClock initialized: now 1714572576363884 us; error 25 us; skew 500 ppm
I20240501 14:09:36.365484 13475 webserver.cc:460] Webserver started at http://127.6.153.198:45069/ using document root <none> and password file <none>
I20240501 14:09:36.365973 13475 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:36.366029 13475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:36.366179 13475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:36.367691 13475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "18bef23ddc624bd784ea38fc6ef00040"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.368115 13475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "18bef23ddc624bd784ea38fc6ef00040"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.370985 13475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:09:36.372939 13489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.373795 13475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:36.373888 13475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "18bef23ddc624bd784ea38fc6ef00040"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.373960 13475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:36.386055 13475 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:36.386541 13475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:36.386675 13475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:36.387111 13475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:36.387950 13475 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:36.387995 13475 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.388042 13475 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:36.388069 13475 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.395509 13475 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.198:38509
I20240501 14:09:36.395541 13603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.198:38509 every 8 connection(s)
I20240501 14:09:36.396303 13475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 14:09:36.400043 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 13475
I20240501 14:09:36.400149 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 14:09:36.402439 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.199:0
--local_ip_for_outbound_sockets=127.6.153.199
--webserver_interface=127.6.153.199
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:36.404619 13604 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:36.404829 13604 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:36.405369 13604 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:36.406359 12751 ts_manager.cc:194] Registered new tserver with Master: 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509)
I20240501 14:09:36.406852 12751 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.198:46393
W20240501 14:09:36.503166 13607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:36.503393 13607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:36.503463 13607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:36.506376 13607 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:36.506436 13607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:36.506520 13607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.199
I20240501 14:09:36.509907 13607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.199:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.6.153.199
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.13607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.199
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:36.510624 13607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:36.511595 13607 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:36.518962 13615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.519351 13613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.519698 13612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:36.520087 13607 server_base.cc:1034] running on GCE node
I20240501 14:09:36.520412 13607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:36.520876 13607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:36.522014 13607 hybrid_clock.cc:648] HybridClock initialized: now 1714572576522002 us; error 23 us; skew 500 ppm
I20240501 14:09:36.523626 13607 webserver.cc:460] Webserver started at http://127.6.153.199:42567/ using document root <none> and password file <none>
I20240501 14:09:36.524102 13607 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:36.524152 13607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:36.524319 13607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:36.525774 13607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "70c63e471e5b482b8b257c3e7543feb5"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.526189 13607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "70c63e471e5b482b8b257c3e7543feb5"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.529139 13607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:09:36.531005 13621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.531886 13607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:36.531992 13607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "70c63e471e5b482b8b257c3e7543feb5"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.532064 13607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:36.547248 12944 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
I20240501 14:09:36.563658 13607 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:36.564172 13607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:36.564316 13607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:36.564772 13607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:36.565563 13607 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:36.565610 13607 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.565658 13607 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:36.565686 13607 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.573165 13607 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.199:38347
I20240501 14:09:36.573187 13735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.199:38347 every 8 connection(s)
I20240501 14:09:36.574010 13607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 14:09:36.577751 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 13607
I20240501 14:09:36.577860 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 14:09:36.579919 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.200:0
--local_ip_for_outbound_sockets=127.6.153.200
--webserver_interface=127.6.153.200
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:36.582334 13736 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:36.582543 13736 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:36.583107 13736 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:36.584115 12751 ts_manager.cc:194] Registered new tserver with Master: 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347)
I20240501 14:09:36.584806 12751 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.199:36929
W20240501 14:09:36.680773 13739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:36.680956 13739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:36.681000 13739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:36.683979 13739 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:36.684033 13739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:36.684099 13739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.200
I20240501 14:09:36.687510 13739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.200:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.6.153.200
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.13739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.200
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:36.688268 13739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:36.689263 13739 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:36.696962 13747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.697261 13744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:36.697702 13745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:36.698132 13739 server_base.cc:1034] running on GCE node
I20240501 14:09:36.698467 13739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:36.698957 13739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:36.700098 13739 hybrid_clock.cc:648] HybridClock initialized: now 1714572576700085 us; error 26 us; skew 500 ppm
I20240501 14:09:36.701870 13739 webserver.cc:460] Webserver started at http://127.6.153.200:43281/ using document root <none> and password file <none>
I20240501 14:09:36.702383 13739 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:36.702438 13739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:36.702590 13739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:36.704186 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "3025ea3ec516466fbe5c5c4337733584"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.704627 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "3025ea3ec516466fbe5c5c4337733584"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.707727 13739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:09:36.709767 13753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.710691 13739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:36.710798 13739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "3025ea3ec516466fbe5c5c4337733584"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:36.710871 13739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:36.720042 13075 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
I20240501 14:09:36.730690 13739 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:36.731196 13739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:36.731362 13739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:36.731832 13739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:36.732635 13739 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:36.732679 13739 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.732729 13739 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:36.732756 13739 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:36.741781 13739 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.200:34743
I20240501 14:09:36.741845 13867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.200:34743 every 8 connection(s)
I20240501 14:09:36.742650 13739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 14:09:36.747140 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 13739
I20240501 14:09:36.747236 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 14:09:36.752241 13868 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:36.752475 13868 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:36.753052 13868 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:36.753983 12751 ts_manager.cc:194] Registered new tserver with Master: 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743)
I20240501 14:09:36.754482 12751 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.200:45193
I20240501 14:09:36.760821 6759 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240501 14:09:36.770351 6759 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240501 14:09:36.770416 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 13343
I20240501 14:09:36.776770 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 13475
I20240501 14:09:36.782860 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 13607
I20240501 14:09:36.787142 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 13739
I20240501 14:09:36.913256 13207 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
I20240501 14:09:37.074380 13340 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
I20240501 14:09:40.542591 12751 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:40.562069 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 34f677061e42436ab08123227f2be7bf (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.563066 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34f677061e42436ab08123227f2be7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.562979 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.563846 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.563963 12876 tablet_service.cc:1467] Processing CreateTablet for tablet 34f677061e42436ab08123227f2be7bf (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.564654 12876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34f677061e42436ab08123227f2be7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.562988 13140 tablet_service.cc:1467] Processing CreateTablet for tablet ecd37c11bf4c45de95f861c1eb4f20d6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.567600 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd37c11bf4c45de95f861c1eb4f20d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.567575 13008 tablet_service.cc:1467] Processing CreateTablet for tablet ecd37c11bf4c45de95f861c1eb4f20d6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.567811 13008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd37c11bf4c45de95f861c1eb4f20d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.562063 13009 tablet_service.cc:1467] Processing CreateTablet for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.569007 13009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.569074 13272 tablet_service.cc:1467] Processing CreateTablet for tablet 34f677061e42436ab08123227f2be7bf (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.569927 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34f677061e42436ab08123227f2be7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.570596 12877 tablet_service.cc:1467] Processing CreateTablet for tablet ecd37c11bf4c45de95f861c1eb4f20d6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.570845 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd37c11bf4c45de95f861c1eb4f20d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.570959 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:40.571190 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:40.572607 13889 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:40.573900 13890 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:40.575171 13889 tablet_bootstrap.cc:654] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.575800 13890 tablet_bootstrap.cc:654] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.576056 13891 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:40.576166 13888 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:40.576082 13889 log.cc:824] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:40.576524 13890 log.cc:824] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:40.577826 13891 tablet_bootstrap.cc:654] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.577894 13889 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:40.577965 13888 tablet_bootstrap.cc:654] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.578032 13890 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:40.578052 13889 ts_tablet_manager.cc:1367] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240501 14:09:40.578172 13890 ts_tablet_manager.cc:1367] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240501 14:09:40.578542 13891 log.cc:824] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:40.578629 13888 log.cc:824] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:40.580127 13888 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:40.580276 13888 ts_tablet_manager.cc:1367] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240501 14:09:40.580988 13891 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:40.581123 13891 ts_tablet_manager.cc:1367] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240501 14:09:40.581331 13889 raft_consensus.cc:348] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.581523 13889 raft_consensus.cc:374] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.581578 13889 raft_consensus.cc:729] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.581754 13890 raft_consensus.cc:348] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.582011 13890 raft_consensus.cc:374] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.582063 13890 raft_consensus.cc:729] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.581981 13889 consensus_queue.cc:260] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.582504 13890 consensus_queue.cc:260] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.582912 13889 ts_tablet_manager.cc:1398] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20240501 14:09:40.583271 13889 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:40.584064 13888 raft_consensus.cc:348] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.584208 13889 tablet_bootstrap.cc:654] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.584317 13888 raft_consensus.cc:374] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.584411 13888 raft_consensus.cc:729] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.584550 13890 ts_tablet_manager.cc:1398] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 14:09:40.584879 13888 consensus_queue.cc:260] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.585680 13888 ts_tablet_manager.cc:1398] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 14:09:40.586035 13888 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:40.585757 13891 raft_consensus.cc:348] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.586679 13891 raft_consensus.cc:374] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.586745 13891 raft_consensus.cc:729] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.587180 13889 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:40.587143 13891 consensus_queue.cc:260] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.587283 13889 ts_tablet_manager.cc:1367] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:40.587534 13890 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:40.587841 13889 raft_consensus.cc:348] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.588017 13889 raft_consensus.cc:374] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.588097 13889 raft_consensus.cc:729] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.588291 13889 consensus_queue.cc:260] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.588544 13888 tablet_bootstrap.cc:654] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.588615 13889 ts_tablet_manager.cc:1398] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:40.588660 13890 tablet_bootstrap.cc:654] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.588914 13889 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:40.589746 13889 tablet_bootstrap.cc:654] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.589183 13891 ts_tablet_manager.cc:1398] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20240501 14:09:40.589877 13888 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:40.589952 13888 ts_tablet_manager.cc:1367] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:40.590046 13891 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:40.590422 13888 raft_consensus.cc:348] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.590514 13888 raft_consensus.cc:374] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.590543 13888 raft_consensus.cc:729] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.590638 13888 consensus_queue.cc:260] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.590885 13891 tablet_bootstrap.cc:654] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:40.590921 13888 ts_tablet_manager.cc:1398] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:40.591214 13889 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:40.591292 13889 ts_tablet_manager.cc:1367] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 14:09:40.591737 13889 raft_consensus.cc:348] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.591835 13889 raft_consensus.cc:374] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.591864 13889 raft_consensus.cc:729] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.591958 13889 consensus_queue.cc:260] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.592190 13889 ts_tablet_manager.cc:1398] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:40.593153 13890 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:40.593228 13890 ts_tablet_manager.cc:1367] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:09:40.593699 13890 raft_consensus.cc:348] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.593806 13890 raft_consensus.cc:374] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.593838 13890 raft_consensus.cc:729] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.593993 13890 consensus_queue.cc:260] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.594301 13890 ts_tablet_manager.cc:1398] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:40.594566 13891 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:40.594648 13891 ts_tablet_manager.cc:1367] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:09:40.595081 13891 raft_consensus.cc:348] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.595161 13891 raft_consensus.cc:374] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:40.595188 13891 raft_consensus.cc:729] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:40.595304 13891 consensus_queue.cc:260] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.595587 13891 ts_tablet_manager.cc:1398] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:40.632395 13897 raft_consensus.cc:482] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:40.632612 13897 raft_consensus.cc:504] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.633746 13897 leader_election.cc:288] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:40.637527 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd37c11bf4c45de95f861c1eb4f20d6" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:40.637562 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd37c11bf4c45de95f861c1eb4f20d6" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:40.637820 13029 raft_consensus.cc:2454] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:40.637820 12897 raft_consensus.cc:2454] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:40.638232 13096 leader_election.cc:302] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:40.638468 13897 raft_consensus.cc:2781] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:40.638540 13897 raft_consensus.cc:482] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:40.638587 13897 raft_consensus.cc:3037] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:40.639878 13897 raft_consensus.cc:504] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.640211 13897 leader_election.cc:288] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:40.640481 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd37c11bf4c45de95f861c1eb4f20d6" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:40.640522 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd37c11bf4c45de95f861c1eb4f20d6" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:40.640599 12897 raft_consensus.cc:3037] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:40.640614 13029 raft_consensus.cc:3037] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:40.641664 12897 raft_consensus.cc:2454] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:40.641968 13029 raft_consensus.cc:2454] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:40.642230 13096 leader_election.cc:302] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:40.642379 13897 raft_consensus.cc:2781] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:40.642554 13897 raft_consensus.cc:686] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:40.642830 13897 consensus_queue.cc:237] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.645570 12753 catalog_manager.cc:5484] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:40.654778 13208 tablet.cc:2369] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:09:40.692919 13900 raft_consensus.cc:482] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:40.693099 13900 raft_consensus.cc:504] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.694257 13900 leader_election.cc:288] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:40.697639 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34f677061e42436ab08123227f2be7bf" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:40.697654 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34f677061e42436ab08123227f2be7bf" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:40.697764 12897 raft_consensus.cc:2454] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:40.697770 13029 raft_consensus.cc:2454] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:40.698122 13227 leader_election.cc:302] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:40.698379 13900 raft_consensus.cc:2781] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:40.698448 13900 raft_consensus.cc:482] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:40.698495 13900 raft_consensus.cc:3037] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:40.699388 13900 raft_consensus.cc:504] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.699678 13900 leader_election.cc:288] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:40.699988 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34f677061e42436ab08123227f2be7bf" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:40.699983 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34f677061e42436ab08123227f2be7bf" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:40.700088 12897 raft_consensus.cc:3037] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:40.700088 13029 raft_consensus.cc:3037] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:40.700789 13029 raft_consensus.cc:2454] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:40.700872 12897 raft_consensus.cc:2454] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:40.701047 13227 leader_election.cc:302] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:40.701201 13900 raft_consensus.cc:2781] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:40.701400 13900 raft_consensus.cc:686] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Leader. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:09:40.701694 13900 consensus_queue.cc:237] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:40.704561 12753 catalog_manager.cc:5484] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 879199fbf918416cb23b0376dde0aa6e (127.6.153.196). New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:40.711548 13076 tablet.cc:2369] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:09:40.788360 12945 tablet.cc:2369] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:09:40.815541 13341 tablet.cc:2369] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:09:40.824731 13900 raft_consensus.cc:482] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:40.824860 13900 raft_consensus.cc:504] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.825304 13900 leader_election.cc:288] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:40.825737 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab8fdef27b64f87b89f6e8e4166e0c2" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:40.825882 13029 raft_consensus.cc:2454] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:40.826171 13227 leader_election.cc:302] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:40.826341 13900 raft_consensus.cc:2781] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:40.826392 13900 raft_consensus.cc:482] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:40.826427 13900 raft_consensus.cc:3037] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:40.827481 13900 raft_consensus.cc:504] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.827807 13900 leader_election.cc:288] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:40.828088 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab8fdef27b64f87b89f6e8e4166e0c2" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:40.828194 13029 raft_consensus.cc:3037] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:40.828840 13029 raft_consensus.cc:2454] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:40.829066 13227 leader_election.cc:302] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:40.829205 13900 raft_consensus.cc:2781] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:40.829177 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab8fdef27b64f87b89f6e8e4166e0c2" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:40.829269 13900 raft_consensus.cc:686] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Leader. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:09:40.829177 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab8fdef27b64f87b89f6e8e4166e0c2" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:40.829418 13161 raft_consensus.cc:2454] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:40.829398 13900 consensus_queue.cc:237] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:40.831049 12753 catalog_manager.cc:5484] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 879199fbf918416cb23b0376dde0aa6e (127.6.153.196). New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } }
W20240501 14:09:40.949296 13914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:40.949502 13914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:40.952422 13914 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 14:09:41.040418 13029 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:41.041446 12897 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:41.041589 13160 raft_consensus.cc:3037] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.041927 13897 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.042487 13897 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.042726 13160 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:41.045776 13028 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:41.046756 13900 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.049187 12897 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:41.050153 13028 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:41.050554 13911 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.050868 13911 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.051118 13911 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.052623 13945 mvcc.cc:205] Tried to move back new op lower bound from 7022889291934973952 to 7022889290315403264. Current Snapshot: MvccSnapshot[applied={T|T < 7022889291934973952}]
I20240501 14:09:41.061548 13940 mvcc.cc:205] Tried to move back new op lower bound from 7022889291958915072 to 7022889291078496256. Current Snapshot: MvccSnapshot[applied={T|T < 7022889291937902592}]
I20240501 14:09:41.068349 13942 mvcc.cc:205] Tried to move back new op lower bound from 7022889292018458624 to 7022889290315403264. Current Snapshot: MvccSnapshot[applied={T|T < 7022889292018458624}]
I20240501 14:09:41.073490 13944 mvcc.cc:205] Tried to move back new op lower bound from 7022889291934973952 to 7022889290315403264. Current Snapshot: MvccSnapshot[applied={T|T < 7022889291934973952}]
INSERT report
rows total: 2000
time total: 119.054 ms
time per row: 0.0595268 ms
I20240501 14:09:41.113605 12752 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:41.118424 13272 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac389d29e274b15b5f3db598a04f5e7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.118508 12877 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac389d29e274b15b5f3db598a04f5e7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.118713 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac389d29e274b15b5f3db598a04f5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.118654 13140 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac389d29e274b15b5f3db598a04f5e7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.118738 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac389d29e274b15b5f3db598a04f5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.118907 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac389d29e274b15b5f3db598a04f5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.120590 13972 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:41.121248 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 8a948907bfb14d0d958ebb969e5a6a50 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.121368 12876 tablet_service.cc:1467] Processing CreateTablet for tablet 8a948907bfb14d0d958ebb969e5a6a50 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.121496 13974 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:41.121498 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a948907bfb14d0d958ebb969e5a6a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.121368 12877 tablet_service.cc:1467] Processing CreateTablet for tablet 37047d45ffe14570b28e459afdb66d7e (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.121702 13972 tablet_bootstrap.cc:654] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.121728 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37047d45ffe14570b28e459afdb66d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.122313 13974 tablet_bootstrap.cc:654] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.122908 13973 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:41.123268 13977 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:41.123481 12876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a948907bfb14d0d958ebb969e5a6a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.123720 13973 tablet_bootstrap.cc:654] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.123975 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 8a948907bfb14d0d958ebb969e5a6a50 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.124048 13977 tablet_bootstrap.cc:654] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.124173 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a948907bfb14d0d958ebb969e5a6a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.124517 13974 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:41.124588 13974 ts_tablet_manager.cc:1367] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:09:41.124994 13974 raft_consensus.cc:348] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.125083 13974 raft_consensus.cc:374] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.125115 13974 raft_consensus.cc:729] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.125252 13974 consensus_queue.cc:260] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.125542 13974 ts_tablet_manager.cc:1398] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.125633 13272 tablet_service.cc:1467] Processing CreateTablet for tablet 37047d45ffe14570b28e459afdb66d7e (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.125726 13974 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:41.125824 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37047d45ffe14570b28e459afdb66d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.126466 13974 tablet_bootstrap.cc:654] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.127130 13140 tablet_service.cc:1467] Processing CreateTablet for tablet 37047d45ffe14570b28e459afdb66d7e (DEFAULT_TABLE table=rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.127362 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37047d45ffe14570b28e459afdb66d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.128198 13973 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:41.128273 13973 ts_tablet_manager.cc:1367] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:09:41.128429 13977 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:41.128513 13977 ts_tablet_manager.cc:1367] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:09:41.128665 13973 raft_consensus.cc:348] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.128839 13973 raft_consensus.cc:374] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.128918 13973 raft_consensus.cc:729] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.129068 13973 consensus_queue.cc:260] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.129127 13977 raft_consensus.cc:348] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.129310 13977 raft_consensus.cc:374] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.129387 13977 raft_consensus.cc:729] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.129441 13973 ts_tablet_manager.cc:1398] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.129549 13977 consensus_queue.cc:260] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.129628 13973 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:41.130416 13972 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:41.130430 13973 tablet_bootstrap.cc:654] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.130497 13972 ts_tablet_manager.cc:1367] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 14:09:41.130932 13972 raft_consensus.cc:348] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.131022 13972 raft_consensus.cc:374] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.131052 13972 raft_consensus.cc:729] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.131155 13974 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:41.131165 13972 consensus_queue.cc:260] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.131222 13974 ts_tablet_manager.cc:1367] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:09:41.131426 13972 ts_tablet_manager.cc:1398] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.131426 13977 ts_tablet_manager.cc:1398] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:09:41.131580 13972 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:41.132221 13974 raft_consensus.cc:348] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.132323 13974 raft_consensus.cc:374] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.132357 13974 raft_consensus.cc:729] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.132388 13972 tablet_bootstrap.cc:654] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.132463 13974 consensus_queue.cc:260] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.132930 13897 raft_consensus.cc:482] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:41.133011 13897 raft_consensus.cc:504] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.133054 13974 ts_tablet_manager.cc:1398] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:09:41.133200 13974 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:41.133415 13897 leader_election.cc:288] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:41.133960 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac389d29e274b15b5f3db598a04f5e7" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:41.134063 12897 raft_consensus.cc:2454] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:41.134369 13094 leader_election.cc:302] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:41.134524 13974 tablet_bootstrap.cc:654] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.134546 13907 raft_consensus.cc:2781] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:41.134593 13907 raft_consensus.cc:482] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:41.134626 13907 raft_consensus.cc:3037] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.135390 13973 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:41.135459 13973 ts_tablet_manager.cc:1367] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:09:41.135509 13907 raft_consensus.cc:504] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.135797 13973 raft_consensus.cc:348] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.135857 13907 leader_election.cc:288] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:41.135880 13973 raft_consensus.cc:374] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.135910 13973 raft_consensus.cc:729] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.136013 13973 consensus_queue.cc:260] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.136297 13973 ts_tablet_manager.cc:1398] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.136706 13974 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:41.136749 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac389d29e274b15b5f3db598a04f5e7" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:41.136821 13974 ts_tablet_manager.cc:1367] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 14:09:41.136845 12897 raft_consensus.cc:3037] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.137183 13974 raft_consensus.cc:348] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.137281 13974 raft_consensus.cc:374] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.137312 13974 raft_consensus.cc:729] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.137435 13974 consensus_queue.cc:260] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.137688 12897 raft_consensus.cc:2454] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:41.137728 13974 ts_tablet_manager.cc:1398] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:41.138149 13094 leader_election.cc:302] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:41.138291 13972 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:41.138337 13907 raft_consensus.cc:2781] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:41.138366 13972 ts_tablet_manager.cc:1367] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 14:09:41.138410 13907 raft_consensus.cc:686] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:41.138520 13907 consensus_queue.cc:237] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:41.138751 13972 raft_consensus.cc:348] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.138835 13972 raft_consensus.cc:374] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.138864 13972 raft_consensus.cc:729] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.138962 13972 consensus_queue.cc:260] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.139199 13972 ts_tablet_manager.cc:1398] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:41.139366 13972 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:41.139838 13292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac389d29e274b15b5f3db598a04f5e7" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:41.139834 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac389d29e274b15b5f3db598a04f5e7" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:41.140017 13292 raft_consensus.cc:3037] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.140200 13972 tablet_bootstrap.cc:654] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.140620 12753 catalog_manager.cc:5484] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:41.140834 13292 raft_consensus.cc:2454] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:41.141140 13972 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:41.141217 13972 ts_tablet_manager.cc:1367] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:41.141593 13972 raft_consensus.cc:348] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.141692 13972 raft_consensus.cc:374] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.141723 13972 raft_consensus.cc:729] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.141825 13972 consensus_queue.cc:260] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.142061 13972 ts_tablet_manager.cc:1398] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.195744 13899 raft_consensus.cc:482] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:41.195865 13899 raft_consensus.cc:504] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.196801 13899 leader_election.cc:288] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:41.201027 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a948907bfb14d0d958ebb969e5a6a50" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:41.201011 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a948907bfb14d0d958ebb969e5a6a50" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:41.201161 13027 raft_consensus.cc:2454] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:41.201161 13160 raft_consensus.cc:2454] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:41.201495 12832 leader_election.cc:302] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:41.201753 13899 raft_consensus.cc:2781] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:41.201814 13899 raft_consensus.cc:482] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:41.201857 13899 raft_consensus.cc:3037] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.202667 13899 raft_consensus.cc:504] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.202981 13899 leader_election.cc:288] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:41.203271 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a948907bfb14d0d958ebb969e5a6a50" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:41.203298 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a948907bfb14d0d958ebb969e5a6a50" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:41.203405 13160 raft_consensus.cc:3037] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.203424 13027 raft_consensus.cc:3037] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.204121 13160 raft_consensus.cc:2454] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:41.204210 13027 raft_consensus.cc:2454] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:41.204402 12831 leader_election.cc:302] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:41.204550 13899 raft_consensus.cc:2781] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:41.204715 13899 raft_consensus.cc:686] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:41.204995 13899 consensus_queue.cc:237] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.207090 12753 catalog_manager.cc:5484] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:41.487643 13899 raft_consensus.cc:482] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:41.487774 13899 raft_consensus.cc:504] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.488253 13899 leader_election.cc:288] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:41.488685 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:41.488838 13160 raft_consensus.cc:2454] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:41.489205 12831 leader_election.cc:302] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:41.489382 13899 raft_consensus.cc:2781] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:41.489449 13899 raft_consensus.cc:482] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:41.489490 13899 raft_consensus.cc:3037] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.490396 13899 raft_consensus.cc:504] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.490691 13899 leader_election.cc:288] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Requested vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:41.491003 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:41.491117 13160 raft_consensus.cc:3037] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.491848 13160 raft_consensus.cc:2454] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:41.492103 12831 leader_election.cc:302] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:41.492334 13899 raft_consensus.cc:2781] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:41.492321 13292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:41.492350 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:41.492419 13899 raft_consensus.cc:686] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:41.492453 13292 raft_consensus.cc:2454] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:41.492553 13899 consensus_queue.cc:237] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:41.494082 12753 catalog_manager.cc:5484] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:41.648845 13951 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.654781 13993 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 14:09:41.679528 13991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:41.679741 13991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:41.682756 13991 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 14:09:41.694005 13899 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.697072 13899 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_1'
I20240501 14:09:41.774554 13293 raft_consensus.cc:3037] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.775693 13293 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:41.776770 13160 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:41.777520 13899 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:41.779223 13899 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 105.941 ms
time per row: 0.0529707 ms
I20240501 14:09:41.828505 12753 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:41.833359 13272 tablet_service.cc:1467] Processing CreateTablet for tablet fe046c56dde144f9adbbbf779067476e (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.833359 12877 tablet_service.cc:1467] Processing CreateTablet for tablet fe046c56dde144f9adbbbf779067476e (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.833477 13007 tablet_service.cc:1467] Processing CreateTablet for tablet fe046c56dde144f9adbbbf779067476e (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.833617 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe046c56dde144f9adbbbf779067476e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.833617 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe046c56dde144f9adbbbf779067476e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.833750 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe046c56dde144f9adbbbf779067476e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.834137 13140 tablet_service.cc:1467] Processing CreateTablet for tablet 78408b0cc50c438caf75876d8dcf88a3 (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.834354 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78408b0cc50c438caf75876d8dcf88a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.836246 13272 tablet_service.cc:1467] Processing CreateTablet for tablet 78408b0cc50c438caf75876d8dcf88a3 (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.836365 12877 tablet_service.cc:1467] Processing CreateTablet for tablet 5053c0f554af4efaa221444af87dd9b9 (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.836470 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78408b0cc50c438caf75876d8dcf88a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.836565 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5053c0f554af4efaa221444af87dd9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.836634 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 5053c0f554af4efaa221444af87dd9b9 (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.836645 13008 tablet_service.cc:1467] Processing CreateTablet for tablet 5053c0f554af4efaa221444af87dd9b9 (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.836833 13008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5053c0f554af4efaa221444af87dd9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.836839 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5053c0f554af4efaa221444af87dd9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.838243 14052 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:41.838827 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 78408b0cc50c438caf75876d8dcf88a3 (DEFAULT_TABLE table=rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:41.838937 14053 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:41.839025 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78408b0cc50c438caf75876d8dcf88a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:41.839223 14051 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:41.839516 14050 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:41.839776 14053 tablet_bootstrap.cc:654] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.840309 14051 tablet_bootstrap.cc:654] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.840489 14050 tablet_bootstrap.cc:654] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.840714 14053 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:41.840795 14053 ts_tablet_manager.cc:1367] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:41.839020 14052 tablet_bootstrap.cc:654] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.841220 14053 raft_consensus.cc:348] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.841311 14053 raft_consensus.cc:374] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.841341 14053 raft_consensus.cc:729] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.841449 14053 consensus_queue.cc:260] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.841712 14053 ts_tablet_manager.cc:1398] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.841866 14052 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:41.841940 14052 ts_tablet_manager.cc:1367] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:41.842263 14052 raft_consensus.cc:348] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:41.842342 14052 raft_consensus.cc:374] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.842362 14052 raft_consensus.cc:729] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.842459 14052 consensus_queue.cc:260] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:41.842706 14052 ts_tablet_manager.cc:1398] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.842922 14052 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:41.843822 14052 tablet_bootstrap.cc:654] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.844678 14051 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:41.844770 14051 ts_tablet_manager.cc:1367] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:09:41.844866 14052 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:41.844936 14052 ts_tablet_manager.cc:1367] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 14:09:41.845147 14050 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:41.845217 14050 ts_tablet_manager.cc:1367] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:09:41.845178 14051 raft_consensus.cc:348] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:41.845265 14051 raft_consensus.cc:374] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.845297 14051 raft_consensus.cc:729] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.845286 14052 raft_consensus.cc:348] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.845362 14052 raft_consensus.cc:374] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.845389 14052 raft_consensus.cc:729] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.845431 14051 consensus_queue.cc:260] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:41.845468 14052 consensus_queue.cc:260] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.845682 14052 ts_tablet_manager.cc:1398] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:41.845734 14050 raft_consensus.cc:348] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:41.845815 14052 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:41.845806 14051 ts_tablet_manager.cc:1398] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.845824 14050 raft_consensus.cc:374] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.845854 14050 raft_consensus.cc:729] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.845944 14051 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:41.846098 14050 consensus_queue.cc:260] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:41.846441 14050 ts_tablet_manager.cc:1398] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.846562 14052 tablet_bootstrap.cc:654] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.846729 14050 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:41.847828 14050 tablet_bootstrap.cc:654] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.848866 14050 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:41.848946 14050 ts_tablet_manager.cc:1367] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:41.849174 14010 raft_consensus.cc:482] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:41.849253 14010 raft_consensus.cc:504] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.849305 14051 tablet_bootstrap.cc:654] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.849328 14050 raft_consensus.cc:348] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.849411 14050 raft_consensus.cc:374] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.849439 14050 raft_consensus.cc:729] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.849534 14050 consensus_queue.cc:260] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.849756 14050 ts_tablet_manager.cc:1398] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.850195 14052 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:41.850270 14052 ts_tablet_manager.cc:1367] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 14:09:41.850350 14051 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:41.850335 14010 leader_election.cc:288] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:41.850416 14051 ts_tablet_manager.cc:1367] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:09:41.850617 14052 raft_consensus.cc:348] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.850710 14052 raft_consensus.cc:374] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.850742 14052 raft_consensus.cc:729] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.850787 14051 raft_consensus.cc:348] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.850839 14052 consensus_queue.cc:260] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.850878 14051 raft_consensus.cc:374] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.850909 14051 raft_consensus.cc:729] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.851011 14051 consensus_queue.cc:260] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.851179 14052 ts_tablet_manager.cc:1398] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:41.851408 14051 ts_tablet_manager.cc:1398] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.851604 14051 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:41.852433 14051 tablet_bootstrap.cc:654] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:41.853291 14051 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:41.853359 14051 ts_tablet_manager.cc:1367] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:41.853758 14051 raft_consensus.cc:348] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.853901 14051 raft_consensus.cc:374] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:41.853979 14051 raft_consensus.cc:729] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:41.854161 14051 consensus_queue.cc:260] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.854517 14051 ts_tablet_manager.cc:1398] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:41.855454 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5053c0f554af4efaa221444af87dd9b9" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:41.855552 12897 raft_consensus.cc:2454] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:41.855901 12962 leader_election.cc:302] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:41.856109 14010 raft_consensus.cc:2781] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:41.856164 14010 raft_consensus.cc:482] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:41.856209 14010 raft_consensus.cc:3037] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.856918 14010 raft_consensus.cc:504] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.857102 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5053c0f554af4efaa221444af87dd9b9" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:41.857195 13293 raft_consensus.cc:2454] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:41.857218 14010 leader_election.cc:288] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:41.857439 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5053c0f554af4efaa221444af87dd9b9" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:41.857532 12897 raft_consensus.cc:3037] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.857637 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5053c0f554af4efaa221444af87dd9b9" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:41.857725 13293 raft_consensus.cc:3037] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:41.858328 13293 raft_consensus.cc:2454] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:41.858525 12897 raft_consensus.cc:2454] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:41.858582 12965 leader_election.cc:302] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:41.858708 14010 raft_consensus.cc:2781] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:41.858868 14010 raft_consensus.cc:686] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Becoming Leader. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Running, Role: LEADER
I20240501 14:09:41.859115 14010 consensus_queue.cc:237] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:41.861120 12753 catalog_manager.cc:5484] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:42.004258 13946 raft_consensus.cc:482] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:42.004367 13946 raft_consensus.cc:504] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.004675 13946 leader_election.cc:288] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:42.004931 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78408b0cc50c438caf75876d8dcf88a3" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:42.005055 13027 raft_consensus.cc:2454] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:42.005038 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78408b0cc50c438caf75876d8dcf88a3" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:42.005169 13161 raft_consensus.cc:2454] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:42.005335 13227 leader_election.cc:302] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:42.005481 13946 raft_consensus.cc:2781] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:42.005538 13946 raft_consensus.cc:482] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:42.005580 13946 raft_consensus.cc:3037] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.006407 13946 raft_consensus.cc:504] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.006682 13946 leader_election.cc:288] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:42.006943 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78408b0cc50c438caf75876d8dcf88a3" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:42.007041 13027 raft_consensus.cc:3037] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.007035 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78408b0cc50c438caf75876d8dcf88a3" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:42.007170 13161 raft_consensus.cc:3037] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.007817 13027 raft_consensus.cc:2454] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:42.007977 13161 raft_consensus.cc:2454] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:42.008085 13227 leader_election.cc:302] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:42.008253 13946 raft_consensus.cc:2781] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:42.008433 13946 raft_consensus.cc:686] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Leader. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:09:42.008558 13946 consensus_queue.cc:237] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.009979 12753 catalog_manager.cc:5484] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 879199fbf918416cb23b0376dde0aa6e (127.6.153.196). New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } }
I20240501 14:09:42.069428 14043 raft_consensus.cc:482] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:42.069554 14043 raft_consensus.cc:504] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.070025 14043 leader_election.cc:288] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:42.070356 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe046c56dde144f9adbbbf779067476e" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:42.070484 13293 raft_consensus.cc:2454] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:42.070452 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe046c56dde144f9adbbbf779067476e" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:42.070581 13027 raft_consensus.cc:2454] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:42.070761 12833 leader_election.cc:302] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:42.070912 14043 raft_consensus.cc:2781] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:42.070971 14043 raft_consensus.cc:482] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:42.071012 14043 raft_consensus.cc:3037] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.071867 14043 raft_consensus.cc:504] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.072162 14043 leader_election.cc:288] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Requested vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:42.072448 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe046c56dde144f9adbbbf779067476e" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:42.072468 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe046c56dde144f9adbbbf779067476e" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:42.072575 13293 raft_consensus.cc:3037] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.072575 13027 raft_consensus.cc:3037] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.073252 13027 raft_consensus.cc:2454] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:42.073437 13293 raft_consensus.cc:2454] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:42.073498 12832 leader_election.cc:302] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:42.073710 14043 raft_consensus.cc:2781] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:42.073809 14043 raft_consensus.cc:686] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:42.073938 14043 consensus_queue.cc:237] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.075472 12753 catalog_manager.cc:5484] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:42.225414 14070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:42.225607 14070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:42.228555 14070 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 14:09:42.249799 14010 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.253516 14010 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.303858 13292 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:42.304391 13027 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:42.304936 13899 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.305277 13899 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.313225 13161 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:42.313695 14067 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.315037 13028 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:42.315606 13946 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 94.0368 ms
time per row: 0.0470184 ms
I20240501 14:09:42.363093 12751 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:42.367735 13140 tablet_service.cc:1467] Processing CreateTablet for tablet bdfa21a72f85470396cc525fba5e379d (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.368003 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdfa21a72f85470396cc525fba5e379d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.367977 13007 tablet_service.cc:1467] Processing CreateTablet for tablet bdfa21a72f85470396cc525fba5e379d (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.368012 12877 tablet_service.cc:1467] Processing CreateTablet for tablet bdfa21a72f85470396cc525fba5e379d (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.368201 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdfa21a72f85470396cc525fba5e379d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.368294 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdfa21a72f85470396cc525fba5e379d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.369861 14123 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:42.370441 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6516b6f77b4389aa53862668486bba (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.370589 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 920be22f0370418885ce155d9874607a (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.370673 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6516b6f77b4389aa53862668486bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.370663 12877 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6516b6f77b4389aa53862668486bba (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.370781 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920be22f0370418885ce155d9874607a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.370843 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6516b6f77b4389aa53862668486bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.370898 14123 tablet_bootstrap.cc:654] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.371877 14123 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:42.371946 14123 ts_tablet_manager.cc:1367] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.372351 14123 raft_consensus.cc:348] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.372440 14123 raft_consensus.cc:374] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.372473 14123 raft_consensus.cc:729] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.372579 14123 consensus_queue.cc:260] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.372865 14123 ts_tablet_manager.cc:1398] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.372849 13272 tablet_service.cc:1467] Processing CreateTablet for tablet 920be22f0370418885ce155d9874607a (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.373000 14124 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:42.373018 14123 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:42.373032 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920be22f0370418885ce155d9874607a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.373739 14124 tablet_bootstrap.cc:654] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.373739 14123 tablet_bootstrap.cc:654] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.374652 14124 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:42.374722 14124 ts_tablet_manager.cc:1367] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.374908 14123 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:42.374980 14123 ts_tablet_manager.cc:1367] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:42.375106 14124 raft_consensus.cc:348] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.375201 14124 raft_consensus.cc:374] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.375231 14124 raft_consensus.cc:729] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.375324 14124 consensus_queue.cc:260] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.375428 13140 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6516b6f77b4389aa53862668486bba (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.375460 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 920be22f0370418885ce155d9874607a (DEFAULT_TABLE table=rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.375586 14124 ts_tablet_manager.cc:1398] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.375659 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920be22f0370418885ce155d9874607a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.376050 14123 raft_consensus.cc:348] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.376140 14123 raft_consensus.cc:374] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.376183 14123 raft_consensus.cc:729] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.376276 14123 consensus_queue.cc:260] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.376431 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6516b6f77b4389aa53862668486bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.376514 14123 ts_tablet_manager.cc:1398] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:42.377460 14124 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:42.377461 14127 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:42.378201 14124 tablet_bootstrap.cc:654] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.378257 14127 tablet_bootstrap.cc:654] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.378860 14125 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:42.379290 14124 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:42.379418 14124 ts_tablet_manager.cc:1367] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:42.379698 14125 tablet_bootstrap.cc:654] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.379770 14124 raft_consensus.cc:348] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.379863 14124 raft_consensus.cc:374] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.379894 14124 raft_consensus.cc:729] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.379992 14124 consensus_queue.cc:260] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.380215 14124 ts_tablet_manager.cc:1398] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.380421 14124 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:42.380713 14127 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:42.380731 14125 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:42.380784 14127 ts_tablet_manager.cc:1367] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:09:42.380834 14125 ts_tablet_manager.cc:1367] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:42.381175 14124 tablet_bootstrap.cc:654] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.381197 14127 raft_consensus.cc:348] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.381224 14125 raft_consensus.cc:348] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.381286 14127 raft_consensus.cc:374] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.381311 14125 raft_consensus.cc:374] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.381317 14127 raft_consensus.cc:729] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.381342 14125 raft_consensus.cc:729] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.381425 14127 consensus_queue.cc:260] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.381510 13946 raft_consensus.cc:482] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:42.381501 14125 consensus_queue.cc:260] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.381561 13946 raft_consensus.cc:504] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.381834 14125 ts_tablet_manager.cc:1398] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.381886 13946 leader_election.cc:288] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:42.382028 14125 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:42.382234 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6516b6f77b4389aa53862668486bba" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:42.382342 12897 raft_consensus.cc:2454] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:42.382831 14124 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:42.382840 14127 ts_tablet_manager.cc:1398] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.382808 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6516b6f77b4389aa53862668486bba" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:42.382869 13225 leader_election.cc:302] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:42.382905 14124 ts_tablet_manager.cc:1367] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:42.382984 14127 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:42.383131 14067 raft_consensus.cc:2781] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:42.383175 14067 raft_consensus.cc:482] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:42.383209 14067 raft_consensus.cc:3037] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.383285 14124 raft_consensus.cc:348] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.383386 14124 raft_consensus.cc:374] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.383419 14124 raft_consensus.cc:729] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.383514 14124 consensus_queue.cc:260] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
W20240501 14:09:42.383572 13226 leader_election.cc:341] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:09:42.383733 14124 ts_tablet_manager.cc:1398] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.383746 14127 tablet_bootstrap.cc:654] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.384114 14125 tablet_bootstrap.cc:654] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.384450 14067 raft_consensus.cc:504] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.384792 14067 leader_election.cc:288] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:42.385197 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6516b6f77b4389aa53862668486bba" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:42.385288 13161 raft_consensus.cc:3037] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.385398 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6516b6f77b4389aa53862668486bba" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:42.385490 12897 raft_consensus.cc:3037] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.385905 14127 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:42.385972 14127 ts_tablet_manager.cc:1367] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:09:42.386088 13161 raft_consensus.cc:2454] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:42.386286 12897 raft_consensus.cc:2454] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:42.386478 13226 leader_election.cc:302] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:42.386533 14127 raft_consensus.cc:348] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.386622 14127 raft_consensus.cc:374] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.386656 14127 raft_consensus.cc:729] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.386683 13946 raft_consensus.cc:2781] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:42.386742 13946 raft_consensus.cc:686] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Leader. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:09:42.386754 14127 consensus_queue.cc:260] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.386835 13946 consensus_queue.cc:237] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.387049 14127 ts_tablet_manager.cc:1398] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.388204 14125 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:42.388289 14125 ts_tablet_manager.cc:1367] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:09:42.388449 12754 catalog_manager.cc:5484] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 879199fbf918416cb23b0376dde0aa6e (127.6.153.196). New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:42.388638 14125 raft_consensus.cc:348] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.388725 14125 raft_consensus.cc:374] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.388748 14125 raft_consensus.cc:729] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.388856 14125 consensus_queue.cc:260] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.389083 14125 ts_tablet_manager.cc:1398] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:42.431085 14105 raft_consensus.cc:482] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:42.431174 14105 raft_consensus.cc:504] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.431567 14105 leader_election.cc:288] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:42.431915 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920be22f0370418885ce155d9874607a" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:42.432037 13291 raft_consensus.cc:2454] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:42.432297 12965 leader_election.cc:302] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:42.432461 14105 raft_consensus.cc:2781] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:42.432530 14105 raft_consensus.cc:482] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:42.432570 14105 raft_consensus.cc:3037] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.433584 14105 raft_consensus.cc:504] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.433933 14105 leader_election.cc:288] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Requested vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:42.434226 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920be22f0370418885ce155d9874607a" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:42.434346 13291 raft_consensus.cc:3037] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.435281 13291 raft_consensus.cc:2454] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:42.435391 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920be22f0370418885ce155d9874607a" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:42.435410 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920be22f0370418885ce155d9874607a" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:42.435528 12965 leader_election.cc:302] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:42.435525 13161 raft_consensus.cc:2454] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:42.435810 14105 raft_consensus.cc:2781] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:42.435892 14105 raft_consensus.cc:686] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Becoming Leader. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Running, Role: LEADER
I20240501 14:09:42.436020 14105 consensus_queue.cc:237] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.437484 12747 catalog_manager.cc:5484] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:42.597191 13951 raft_consensus.cc:482] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:42.597307 13951 raft_consensus.cc:504] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.597704 13951 leader_election.cc:288] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:42.598052 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdfa21a72f85470396cc525fba5e379d" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:42.598047 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdfa21a72f85470396cc525fba5e379d" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:42.598171 12897 raft_consensus.cc:2454] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:42.598178 13027 raft_consensus.cc:2454] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:42.598454 13094 leader_election.cc:302] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:42.598634 13951 raft_consensus.cc:2781] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:42.598696 13951 raft_consensus.cc:482] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:42.598737 13951 raft_consensus.cc:3037] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.599717 13951 raft_consensus.cc:504] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.600016 13951 leader_election.cc:288] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:42.600325 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdfa21a72f85470396cc525fba5e379d" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:42.600317 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdfa21a72f85470396cc525fba5e379d" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:42.600422 13027 raft_consensus.cc:3037] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.600422 12897 raft_consensus.cc:3037] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.601104 13027 raft_consensus.cc:2454] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:42.601111 12897 raft_consensus.cc:2454] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:42.601341 13096 leader_election.cc:302] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:42.601524 13951 raft_consensus.cc:2781] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:42.601722 13951 raft_consensus.cc:686] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:42.601864 13951 consensus_queue.cc:237] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:42.603421 12747 catalog_manager.cc:5484] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:42.759708 14143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:42.759897 14143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:42.762995 14143 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 14:09:42.832707 14105 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.835415 13161 raft_consensus.cc:3037] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:42.836724 14065 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.849822 12897 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:42.850271 13951 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.850764 13161 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:42.851272 14136 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.851696 12897 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:42.852051 14136 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:42.852505 13027 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:42.853799 14141 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 104.175 ms
time per row: 0.0520873 ms
I20240501 14:09:42.907485 12753 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:42.912287 12877 tablet_service.cc:1467] Processing CreateTablet for tablet f63509c2039d49f09eee28a45e03ad81 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.912392 13007 tablet_service.cc:1467] Processing CreateTablet for tablet f63509c2039d49f09eee28a45e03ad81 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.912544 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63509c2039d49f09eee28a45e03ad81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.912647 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63509c2039d49f09eee28a45e03ad81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.914207 14196 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:42.914578 13140 tablet_service.cc:1467] Processing CreateTablet for tablet f63509c2039d49f09eee28a45e03ad81 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.914800 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63509c2039d49f09eee28a45e03ad81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.914999 14196 tablet_bootstrap.cc:654] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.916006 14196 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:42.916074 14196 ts_tablet_manager.cc:1367] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.916622 14196 raft_consensus.cc:348] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.916718 14196 raft_consensus.cc:374] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.916759 14196 raft_consensus.cc:729] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.916790 14199 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:42.916893 14196 consensus_queue.cc:260] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.916934 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 7e34fd488c214188aac9990af9de0db4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.917130 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e34fd488c214188aac9990af9de0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.917229 14196 ts_tablet_manager.cc:1398] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.917562 14199 tablet_bootstrap.cc:654] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.918457 14199 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:42.918521 14199 ts_tablet_manager.cc:1367] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.918881 14199 raft_consensus.cc:348] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.918962 14199 raft_consensus.cc:374] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.918987 14199 raft_consensus.cc:729] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.919116 14199 consensus_queue.cc:260] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.919431 14199 ts_tablet_manager.cc:1398] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.919432 12876 tablet_service.cc:1467] Processing CreateTablet for tablet 7e34fd488c214188aac9990af9de0db4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.919432 12877 tablet_service.cc:1467] Processing CreateTablet for tablet d8ea7bbf6eec49728d69ec4b6e825e1e (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.919683 12876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e34fd488c214188aac9990af9de0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.919839 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8ea7bbf6eec49728d69ec4b6e825e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.920033 14199 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:42.920390 13272 tablet_service.cc:1467] Processing CreateTablet for tablet d8ea7bbf6eec49728d69ec4b6e825e1e (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.920634 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8ea7bbf6eec49728d69ec4b6e825e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.920814 14199 tablet_bootstrap.cc:654] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.922499 14202 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:42.922648 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 7e34fd488c214188aac9990af9de0db4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.922845 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e34fd488c214188aac9990af9de0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.923274 14202 tablet_bootstrap.cc:654] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.926435 13007 tablet_service.cc:1467] Processing CreateTablet for tablet d8ea7bbf6eec49728d69ec4b6e825e1e (DEFAULT_TABLE table=rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:42.926640 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8ea7bbf6eec49728d69ec4b6e825e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:42.927538 14197 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:42.928241 14196 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:42.928298 14197 tablet_bootstrap.cc:654] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.928352 14202 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:42.928424 14202 ts_tablet_manager.cc:1367] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:09:42.928844 14202 raft_consensus.cc:348] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.928931 14199 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:42.928932 14202 raft_consensus.cc:374] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.928963 14202 raft_consensus.cc:729] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.928995 14199 ts_tablet_manager.cc:1367] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:09:42.929019 14196 tablet_bootstrap.cc:654] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.929082 14202 consensus_queue.cc:260] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.929337 14202 ts_tablet_manager.cc:1398] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:42.929388 14199 raft_consensus.cc:348] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.929467 14199 raft_consensus.cc:374] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.929487 14202 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:42.929503 14199 raft_consensus.cc:729] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.929631 14199 consensus_queue.cc:260] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.930223 14196 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:42.930261 14202 tablet_bootstrap.cc:654] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.930315 14196 ts_tablet_manager.cc:1367] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:42.930716 14196 raft_consensus.cc:348] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.930800 14196 raft_consensus.cc:374] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.930830 14196 raft_consensus.cc:729] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.930930 14196 consensus_queue.cc:260] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.931169 14196 ts_tablet_manager.cc:1398] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.931185 14202 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:42.931252 14202 ts_tablet_manager.cc:1367] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.931671 14202 raft_consensus.cc:348] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.931758 14202 raft_consensus.cc:374] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.931787 14202 raft_consensus.cc:729] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.931895 14202 consensus_queue.cc:260] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.932097 14199 ts_tablet_manager.cc:1398] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.932132 14202 ts_tablet_manager.cc:1398] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:42.932756 14197 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:42.932873 14197 ts_tablet_manager.cc:1367] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 14:09:42.933422 14197 raft_consensus.cc:348] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.933538 14197 raft_consensus.cc:374] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.933570 14197 raft_consensus.cc:729] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.933737 14197 consensus_queue.cc:260] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:42.934007 14197 ts_tablet_manager.cc:1398] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.934155 14197 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:42.934937 14197 tablet_bootstrap.cc:654] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.936022 14197 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:42.936094 14197 ts_tablet_manager.cc:1367] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.936421 14197 raft_consensus.cc:348] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.936508 14197 raft_consensus.cc:374] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.936549 14197 raft_consensus.cc:729] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.936668 14197 consensus_queue.cc:260] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.936897 14197 ts_tablet_manager.cc:1398] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:42.937055 14197 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:42.937829 14197 tablet_bootstrap.cc:654] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:42.938773 14197 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:42.938858 14197 ts_tablet_manager.cc:1367] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:42.939261 14197 raft_consensus.cc:348] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.939363 14197 raft_consensus.cc:374] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:42.939399 14197 raft_consensus.cc:729] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:42.939500 14197 consensus_queue.cc:260] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.939716 14197 ts_tablet_manager.cc:1398] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:42.998225 13951 raft_consensus.cc:482] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:42.998342 13951 raft_consensus.cc:504] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:42.998704 13951 leader_election.cc:288] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:42.999082 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:42.999115 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:42.999215 12896 raft_consensus.cc:2454] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:42.999245 13291 raft_consensus.cc:2454] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:42.999517 13094 leader_election.cc:302] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:42.999667 13951 raft_consensus.cc:2781] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:42.999724 13951 raft_consensus.cc:482] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:42.999773 13951 raft_consensus.cc:3037] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.000552 13951 raft_consensus.cc:504] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.000885 13951 leader_election.cc:288] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:43.001158 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:43.001264 12896 raft_consensus.cc:3037] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.001258 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:43.001366 13291 raft_consensus.cc:3037] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.002084 12896 raft_consensus.cc:2454] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.002204 13291 raft_consensus.cc:2454] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.002327 13094 leader_election.cc:302] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.002456 13951 raft_consensus.cc:2781] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:43.002517 13951 raft_consensus.cc:686] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:43.002638 13951 consensus_queue.cc:237] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.004060 12747 catalog_manager.cc:5484] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:43.011098 13951 raft_consensus.cc:482] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:43.011171 13951 raft_consensus.cc:504] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.011463 13951 leader_election.cc:288] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:43.011744 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63509c2039d49f09eee28a45e03ad81" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:43.011797 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63509c2039d49f09eee28a45e03ad81" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:43.011840 12896 raft_consensus.cc:2454] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:43.011929 13027 raft_consensus.cc:2454] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:43.012085 13094 leader_election.cc:302] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.012229 13951 raft_consensus.cc:2781] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:43.012279 13951 raft_consensus.cc:482] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:43.012305 13951 raft_consensus.cc:3037] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.012926 13951 raft_consensus.cc:504] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.013200 13951 leader_election.cc:288] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:43.013458 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63509c2039d49f09eee28a45e03ad81" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:43.013500 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63509c2039d49f09eee28a45e03ad81" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:43.013569 12896 raft_consensus.cc:3037] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.013590 13027 raft_consensus.cc:3037] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.014314 12896 raft_consensus.cc:2454] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.014410 13027 raft_consensus.cc:2454] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.014546 13094 leader_election.cc:302] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.014675 13951 raft_consensus.cc:2781] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:43.014755 13951 raft_consensus.cc:686] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:43.014856 13951 consensus_queue.cc:237] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.016323 12747 catalog_manager.cc:5484] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } }
I20240501 14:09:43.273105 14065 raft_consensus.cc:482] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:43.273216 14065 raft_consensus.cc:504] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.273550 14065 leader_election.cc:288] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:43.273869 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8ea7bbf6eec49728d69ec4b6e825e1e" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:43.273871 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8ea7bbf6eec49728d69ec4b6e825e1e" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:43.273985 12897 raft_consensus.cc:2454] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:43.273993 13291 raft_consensus.cc:2454] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:43.274228 12965 leader_election.cc:302] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:43.274381 14065 raft_consensus.cc:2781] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:43.274434 14065 raft_consensus.cc:482] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:43.274475 14065 raft_consensus.cc:3037] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.275221 14065 raft_consensus.cc:504] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.275530 14065 leader_election.cc:288] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:43.275811 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8ea7bbf6eec49728d69ec4b6e825e1e" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:43.275843 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8ea7bbf6eec49728d69ec4b6e825e1e" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:43.275929 13291 raft_consensus.cc:3037] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.275925 12897 raft_consensus.cc:3037] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.276762 12897 raft_consensus.cc:2454] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:43.276765 13291 raft_consensus.cc:2454] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:43.277019 12965 leader_election.cc:302] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:43.277156 14065 raft_consensus.cc:2781] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:43.277240 14065 raft_consensus.cc:686] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Becoming Leader. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Running, Role: LEADER
I20240501 14:09:43.277369 14065 consensus_queue.cc:237] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.278909 12747 catalog_manager.cc:5484] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:43.379256 13951 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:43.383298 14141 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 14:09:43.387538 14213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:43.387727 14213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:43.390635 14213 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240501 14:09:43.475381 13027 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:43.475615 12897 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:43.475775 14141 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:43.476079 14141 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:43.480376 13291 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:43.480923 14065 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:43.491946 12897 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:43.492269 14065 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 113.458 ms
time per row: 0.0567288 ms
I20240501 14:09:43.542181 12748 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:43.547050 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 439e22f89e944051b44d3263c43b2fc3 (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.547360 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 439e22f89e944051b44d3263c43b2fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.547937 12877 tablet_service.cc:1467] Processing CreateTablet for tablet c2affe37c7174ff1be13ba4807d1fffe (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.548018 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 439e22f89e944051b44d3263c43b2fc3 (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.548182 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2affe37c7174ff1be13ba4807d1fffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.548254 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 439e22f89e944051b44d3263c43b2fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.550019 14273 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:43.550320 13272 tablet_service.cc:1467] Processing CreateTablet for tablet 9d696c814eaa4c718e49dc32df0d6100 (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.550509 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d696c814eaa4c718e49dc32df0d6100. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.550522 14272 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:43.550772 14273 tablet_bootstrap.cc:654] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.551409 14272 tablet_bootstrap.cc:654] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.551760 14273 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:43.551828 14273 ts_tablet_manager.cc:1367] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:43.552161 14273 raft_consensus.cc:348] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.552240 14273 raft_consensus.cc:374] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.552260 14273 raft_consensus.cc:729] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.552353 14273 consensus_queue.cc:260] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.552605 14273 ts_tablet_manager.cc:1398] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:43.552940 13140 tablet_service.cc:1467] Processing CreateTablet for tablet c2affe37c7174ff1be13ba4807d1fffe (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.553103 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2affe37c7174ff1be13ba4807d1fffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.553303 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 9d696c814eaa4c718e49dc32df0d6100 (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.553501 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d696c814eaa4c718e49dc32df0d6100. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.556138 13008 tablet_service.cc:1467] Processing CreateTablet for tablet 9d696c814eaa4c718e49dc32df0d6100 (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.556396 13008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d696c814eaa4c718e49dc32df0d6100. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.556418 13009 tablet_service.cc:1467] Processing CreateTablet for tablet c2affe37c7174ff1be13ba4807d1fffe (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.556138 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 439e22f89e944051b44d3263c43b2fc3 (DEFAULT_TABLE table=rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:43.556689 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 439e22f89e944051b44d3263c43b2fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.556948 14274 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:43.557675 14274 tablet_bootstrap.cc:654] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.558173 13009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2affe37c7174ff1be13ba4807d1fffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:43.558923 14274 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:43.559439 14274 ts_tablet_manager.cc:1367] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:43.559481 14272 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:43.559548 14272 ts_tablet_manager.cc:1367] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240501 14:09:43.560050 14274 raft_consensus.cc:348] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.560175 14274 raft_consensus.cc:374] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.560180 14272 raft_consensus.cc:348] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.560272 14272 raft_consensus.cc:374] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.560256 14274 raft_consensus.cc:729] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.560312 14272 raft_consensus.cc:729] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.560395 14274 consensus_queue.cc:260] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.560421 14272 consensus_queue.cc:260] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.560657 14274 ts_tablet_manager.cc:1398] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:43.560804 14274 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:43.561048 14272 ts_tablet_manager.cc:1398] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:43.561208 14272 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:43.561998 14272 tablet_bootstrap.cc:654] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.562688 14278 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:43.563063 14274 tablet_bootstrap.cc:654] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.563639 14278 tablet_bootstrap.cc:654] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.564054 14274 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:43.564175 14274 ts_tablet_manager.cc:1367] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 14:09:43.564437 14272 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:43.564514 14272 ts_tablet_manager.cc:1367] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 14:09:43.564594 14278 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:43.564604 14274 raft_consensus.cc:348] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.564668 14278 ts_tablet_manager.cc:1367] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:43.564693 14274 raft_consensus.cc:374] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.564720 14274 raft_consensus.cc:729] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.564829 14274 consensus_queue.cc:260] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.564911 14272 raft_consensus.cc:348] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.564999 14272 raft_consensus.cc:374] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.565029 14272 raft_consensus.cc:729] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.565055 14278 raft_consensus.cc:348] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.565126 14274 ts_tablet_manager.cc:1398] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:43.565157 14278 raft_consensus.cc:374] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.565130 14272 consensus_queue.cc:260] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.565187 14278 raft_consensus.cc:729] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.565294 14278 consensus_queue.cc:260] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.565371 14272 ts_tablet_manager.cc:1398] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:43.565519 14272 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:43.565531 14278 ts_tablet_manager.cc:1398] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:43.565652 14278 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:43.566234 14272 tablet_bootstrap.cc:654] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.566402 14278 tablet_bootstrap.cc:654] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.567267 14278 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:43.567449 14272 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:43.567456 14278 ts_tablet_manager.cc:1367] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:43.567523 14272 ts_tablet_manager.cc:1367] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 14:09:43.567833 14278 raft_consensus.cc:348] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.567919 14278 raft_consensus.cc:374] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.567947 14278 raft_consensus.cc:729] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.567909 14272 raft_consensus.cc:348] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.567992 14272 raft_consensus.cc:374] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.568023 14272 raft_consensus.cc:729] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.568068 14278 consensus_queue.cc:260] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.568323 14278 ts_tablet_manager.cc:1398] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:43.568490 14278 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:43.569156 14272 consensus_queue.cc:260] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.569252 14278 tablet_bootstrap.cc:654] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:43.569528 14272 ts_tablet_manager.cc:1398] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:43.570423 14278 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:43.570497 14278 ts_tablet_manager.cc:1367] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:43.570892 14278 raft_consensus.cc:348] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.570991 14278 raft_consensus.cc:374] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:43.571028 14278 raft_consensus.cc:729] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:43.571211 14278 consensus_queue.cc:260] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.571506 14278 ts_tablet_manager.cc:1398] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:43.776000 14251 raft_consensus.cc:482] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:43.776140 14251 raft_consensus.cc:504] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.776548 14251 leader_election.cc:288] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:43.776917 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:43.776940 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:43.777037 13027 raft_consensus.cc:2454] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:43.777079 13291 raft_consensus.cc:2454] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:43.777294 13096 leader_election.cc:302] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.777479 14251 raft_consensus.cc:2781] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:43.777540 14251 raft_consensus.cc:482] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:43.777566 14251 raft_consensus.cc:3037] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.778403 14251 raft_consensus.cc:504] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.778702 14251 leader_election.cc:288] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:43.779004 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:43.779004 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:43.779119 13291 raft_consensus.cc:3037] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.779119 13027 raft_consensus.cc:3037] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.780030 13291 raft_consensus.cc:2454] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.780030 13027 raft_consensus.cc:2454] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.780337 13097 leader_election.cc:302] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.780493 14251 raft_consensus.cc:2781] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:43.780572 14251 raft_consensus.cc:686] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:43.780704 14251 consensus_queue.cc:237] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:43.782347 12748 catalog_manager.cc:5484] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:43.787556 14251 raft_consensus.cc:482] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:43.787628 14251 raft_consensus.cc:504] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.787899 14251 leader_election.cc:288] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:43.788194 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d696c814eaa4c718e49dc32df0d6100" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:43.788280 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d696c814eaa4c718e49dc32df0d6100" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:43.788311 13027 raft_consensus.cc:2454] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:43.788375 13291 raft_consensus.cc:2454] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:43.788524 13096 leader_election.cc:302] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.788708 14251 raft_consensus.cc:2781] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:43.788751 14251 raft_consensus.cc:482] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:43.788781 14251 raft_consensus.cc:3037] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.815948 14251 raft_consensus.cc:504] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.816254 14251 leader_election.cc:288] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:43.816529 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d696c814eaa4c718e49dc32df0d6100" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:43.816589 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d696c814eaa4c718e49dc32df0d6100" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:43.816648 13027 raft_consensus.cc:3037] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.816679 13291 raft_consensus.cc:3037] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.817493 13027 raft_consensus.cc:2454] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.817493 13291 raft_consensus.cc:2454] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.817760 13096 leader_election.cc:302] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.817907 14251 raft_consensus.cc:2781] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:43.817987 14251 raft_consensus.cc:686] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:43.818149 14251 consensus_queue.cc:237] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.819625 12748 catalog_manager.cc:5484] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } }
I20240501 14:09:43.821734 14251 raft_consensus.cc:482] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:43.821801 14251 raft_consensus.cc:504] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.822039 14251 leader_election.cc:288] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:43.822335 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:43.822435 13027 raft_consensus.cc:2454] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:43.822400 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:43.822527 12896 raft_consensus.cc:2454] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:43.822675 13096 leader_election.cc:302] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.822867 14251 raft_consensus.cc:2781] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:43.822909 14251 raft_consensus.cc:482] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:43.822947 14251 raft_consensus.cc:3037] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.823580 14251 raft_consensus.cc:504] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.823880 14251 leader_election.cc:288] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:43.824101 13027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:43.824206 13027 raft_consensus.cc:3037] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.824193 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:43.824283 12896 raft_consensus.cc:3037] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:43.824826 13027 raft_consensus.cc:2454] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.825084 13096 leader_election.cc:302] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:43.825270 14251 raft_consensus.cc:2781] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:43.825331 12896 raft_consensus.cc:2454] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:43.825377 14251 raft_consensus.cc:686] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:43.825484 14251 consensus_queue.cc:237] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:43.826808 12748 catalog_manager.cc:5484] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } }
W20240501 14:09:43.943827 14289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:43.944022 14289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:43.946926 14289 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240501 14:09:44.014382 12896 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.015573 13029 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.015674 13028 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.016137 14251 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.015573 13027 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.017709 13293 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.018038 14251 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.018162 13291 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.018047 14215 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.018349 14251 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.018970 14215 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.019196 14319 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 81.7559 ms
time per row: 0.0408779 ms
I20240501 14:09:44.066601 12747 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:44.079232 12877 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca90427253c4320b2fea78e3fc89d92 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.079560 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca90427253c4320b2fea78e3fc89d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.079514 13272 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca90427253c4320b2fea78e3fc89d92 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.079799 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca90427253c4320b2fea78e3fc89d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.080165 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 627fd32202d247c2af434ab8be152102 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.080432 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627fd32202d247c2af434ab8be152102. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.080653 12876 tablet_service.cc:1467] Processing CreateTablet for tablet 627fd32202d247c2af434ab8be152102 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.080893 12876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627fd32202d247c2af434ab8be152102. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.081317 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 627fd32202d247c2af434ab8be152102 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.081539 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627fd32202d247c2af434ab8be152102. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.081980 13271 tablet_service.cc:1467] Processing CreateTablet for tablet 5583f4e9fe9741649d365daa521355c8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.082223 13271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5583f4e9fe9741649d365daa521355c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.082634 14346 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:44.082819 12875 tablet_service.cc:1467] Processing CreateTablet for tablet 5583f4e9fe9741649d365daa521355c8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.082929 13009 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca90427253c4320b2fea78e3fc89d92 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.083068 12875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5583f4e9fe9741649d365daa521355c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.083156 13009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca90427253c4320b2fea78e3fc89d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.083505 14346 tablet_bootstrap.cc:654] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.084372 14346 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:44.084451 14346 ts_tablet_manager.cc:1367] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.084779 14346 raft_consensus.cc:348] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.084867 14346 raft_consensus.cc:374] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.084894 14346 raft_consensus.cc:729] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.085032 14346 consensus_queue.cc:260] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.085378 14346 ts_tablet_manager.cc:1398] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.085353 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 5583f4e9fe9741649d365daa521355c8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.085552 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5583f4e9fe9741649d365daa521355c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.087782 14348 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:44.088510 14348 tablet_bootstrap.cc:654] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.089395 14348 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:44.089486 14348 ts_tablet_manager.cc:1367] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.089835 14348 raft_consensus.cc:348] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.089917 14348 raft_consensus.cc:374] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.089937 14348 raft_consensus.cc:729] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.090046 14348 consensus_queue.cc:260] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.090284 14348 ts_tablet_manager.cc:1398] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:44.090425 14348 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:44.091118 14348 tablet_bootstrap.cc:654] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.092094 14348 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:44.092160 14348 ts_tablet_manager.cc:1367] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.092468 14348 raft_consensus.cc:348] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.092543 14348 raft_consensus.cc:374] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.092574 14348 raft_consensus.cc:729] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.092684 14348 consensus_queue.cc:260] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.092922 14348 ts_tablet_manager.cc:1398] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:44.097378 14269 raft_consensus.cc:482] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:44.097456 14269 raft_consensus.cc:504] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.097781 14269 leader_election.cc:288] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
W20240501 14:09:44.098426 12965 leader_election.cc:341] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): Not found: Tablet not found: 3ca90427253c4320b2fea78e3fc89d92
W20240501 14:09:44.098551 12962 leader_election.cc:341] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): Not found: Tablet not found: 3ca90427253c4320b2fea78e3fc89d92
I20240501 14:09:44.098611 12962 leader_election.cc:302] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff; no voters: 879199fbf918416cb23b0376dde0aa6e, a3d67dc22c8a4cc2ba08b836537105e7
I20240501 14:09:44.098719 14269 raft_consensus.cc:2726] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 14:09:44.102722 14352 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:44.103006 14351 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:44.103533 14352 tablet_bootstrap.cc:654] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.103789 14351 tablet_bootstrap.cc:654] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.104437 14352 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:44.104518 14352 ts_tablet_manager.cc:1367] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.104719 14351 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:44.104799 14351 ts_tablet_manager.cc:1367] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.104926 14352 raft_consensus.cc:348] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.105020 14352 raft_consensus.cc:374] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.105057 14352 raft_consensus.cc:729] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.105177 14352 consensus_queue.cc:260] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.105193 14351 raft_consensus.cc:348] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.105270 14351 raft_consensus.cc:374] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.105307 14351 raft_consensus.cc:729] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.105418 14351 consensus_queue.cc:260] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.105566 14352 ts_tablet_manager.cc:1398] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.105707 14352 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:44.105706 14351 ts_tablet_manager.cc:1398] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.105866 14351 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:44.106460 14352 tablet_bootstrap.cc:654] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.106572 14351 tablet_bootstrap.cc:654] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.107386 14352 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:44.107457 14352 ts_tablet_manager.cc:1367] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:44.107834 14352 raft_consensus.cc:348] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.107939 14352 raft_consensus.cc:374] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.107969 14352 raft_consensus.cc:729] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.108086 14352 consensus_queue.cc:260] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.108300 14352 ts_tablet_manager.cc:1398] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:44.108433 14352 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:44.109117 14352 tablet_bootstrap.cc:654] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.109732 14327 raft_consensus.cc:482] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:44.109812 14327 raft_consensus.cc:504] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.110113 14352 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:44.110162 14327 leader_election.cc:288] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:44.110185 14352 ts_tablet_manager.cc:1367] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.110417 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5583f4e9fe9741649d365daa521355c8" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:44.110513 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5583f4e9fe9741649d365daa521355c8" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:44.110615 13029 raft_consensus.cc:2454] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:44.110714 14352 raft_consensus.cc:348] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.110795 14352 raft_consensus.cc:374] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.110822 14352 raft_consensus.cc:729] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
W20240501 14:09:44.110833 13225 leader_election.cc:341] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:09:44.110917 14352 consensus_queue.cc:260] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.110940 13227 leader_election.cc:302] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters: a3d67dc22c8a4cc2ba08b836537105e7
I20240501 14:09:44.111105 14136 raft_consensus.cc:2781] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:44.111150 14136 raft_consensus.cc:482] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:44.111187 14136 raft_consensus.cc:3037] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.111645 14351 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:44.111725 14351 ts_tablet_manager.cc:1367] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:09:44.112015 14136 raft_consensus.cc:504] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.112110 14351 raft_consensus.cc:348] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.112224 14351 raft_consensus.cc:374] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.112254 14351 raft_consensus.cc:729] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.112330 14136 leader_election.cc:288] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:44.112376 14351 consensus_queue.cc:260] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.112506 14352 ts_tablet_manager.cc:1398] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.112593 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5583f4e9fe9741649d365daa521355c8" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:44.112627 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5583f4e9fe9741649d365daa521355c8" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:44.112731 13029 raft_consensus.cc:3037] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.113340 13029 raft_consensus.cc:2454] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:44.113538 12896 raft_consensus.cc:3037] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.113626 13227 leader_election.cc:302] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:44.113734 14351 ts_tablet_manager.cc:1398] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.113801 14136 raft_consensus.cc:2781] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:44.113863 14136 raft_consensus.cc:686] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Leader. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:09:44.113906 14351 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:44.113963 14136 consensus_queue.cc:237] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.114194 12896 raft_consensus.cc:2454] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:44.114660 14351 tablet_bootstrap.cc:654] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.115839 14351 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:44.115906 14351 ts_tablet_manager.cc:1367] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.116233 14351 raft_consensus.cc:348] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.116305 14351 raft_consensus.cc:374] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.116333 14351 raft_consensus.cc:729] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.116271 12747 catalog_manager.cc:5484] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 879199fbf918416cb23b0376dde0aa6e (127.6.153.196). New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:44.116446 14351 consensus_queue.cc:260] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.116711 14351 ts_tablet_manager.cc:1398] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.130646 13899 raft_consensus.cc:482] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:44.130722 13899 raft_consensus.cc:504] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.130999 13899 leader_election.cc:288] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:44.131253 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fd32202d247c2af434ab8be152102" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:44.131373 13293 raft_consensus.cc:2454] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:44.131366 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fd32202d247c2af434ab8be152102" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:44.131492 13161 raft_consensus.cc:2454] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:44.131587 12833 leader_election.cc:302] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:44.131718 13899 raft_consensus.cc:2781] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:44.131767 13899 raft_consensus.cc:482] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:44.131793 13899 raft_consensus.cc:3037] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.132395 13899 raft_consensus.cc:504] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.132689 13899 leader_election.cc:288] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Requested vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:44.132916 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fd32202d247c2af434ab8be152102" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:44.133018 13293 raft_consensus.cc:3037] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.132993 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fd32202d247c2af434ab8be152102" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:44.133086 13161 raft_consensus.cc:3037] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.133812 13293 raft_consensus.cc:2454] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:44.133900 13161 raft_consensus.cc:2454] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:44.134110 12833 leader_election.cc:302] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:44.134272 13899 raft_consensus.cc:2781] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:44.134428 13899 raft_consensus.cc:686] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:44.134542 13899 consensus_queue.cc:237] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:44.135871 12747 catalog_manager.cc:5484] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:44.164033 13899 raft_consensus.cc:482] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:44.164120 13899 raft_consensus.cc:504] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.164414 13899 leader_election.cc:288] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:44.164705 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca90427253c4320b2fea78e3fc89d92" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:44.164737 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca90427253c4320b2fea78e3fc89d92" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:44.164819 13293 raft_consensus.cc:2454] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:44.164842 13029 raft_consensus.cc:2454] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:44.165050 12833 leader_election.cc:302] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:44.165196 13899 raft_consensus.cc:2781] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:44.165244 13899 raft_consensus.cc:482] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:44.165279 13899 raft_consensus.cc:3037] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.165907 13899 raft_consensus.cc:504] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.166175 13899 leader_election.cc:288] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:44.166424 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca90427253c4320b2fea78e3fc89d92" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:44.166443 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca90427253c4320b2fea78e3fc89d92" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:44.166507 13029 raft_consensus.cc:3037] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.166530 13293 raft_consensus.cc:3037] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.167145 13293 raft_consensus.cc:2454] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:44.167147 13029 raft_consensus.cc:2454] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:44.167398 12832 leader_election.cc:302] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:44.167534 13899 raft_consensus.cc:2781] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:44.167610 13899 raft_consensus.cc:686] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:44.167730 13899 consensus_queue.cc:237] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.169075 12747 catalog_manager.cc:5484] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:44.275218 14364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:44.275457 14364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:44.278386 14364 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240501 14:09:44.372778 13029 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.372846 13160 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.373139 13029 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.373203 14361 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.373311 13899 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.373512 14136 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.374217 13293 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.374786 13291 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.375742 13899 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.376029 13899 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.381641 14318 mvcc.cc:205] Tried to move back new op lower bound from 7022889305586585600 to 7022889304616005632. Current Snapshot: MvccSnapshot[applied={T|T < 7022889305586585600}]
I20240501 14:09:44.396248 12896 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.396685 14327 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.403174 14322 mvcc.cc:205] Tried to move back new op lower bound from 7022889305587335168 to 7022889304752001024. Current Snapshot: MvccSnapshot[applied={T|T < 7022889305587335168}]
INSERT report
rows total: 2000
time total: 110.287 ms
time per row: 0.0551433 ms
I20240501 14:09:44.419027 14329 mvcc.cc:205] Tried to move back new op lower bound from 7022889305600073728 to 7022889304616005632. Current Snapshot: MvccSnapshot[applied={T|T < 7022889305586585600}]
I20240501 14:09:44.430124 12748 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:44.434873 12876 tablet_service.cc:1467] Processing CreateTablet for tablet 73407db0172f454baf2bb44a85add182 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.434979 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 73407db0172f454baf2bb44a85add182 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.435142 12876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73407db0172f454baf2bb44a85add182. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.435189 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73407db0172f454baf2bb44a85add182. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.435572 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 73407db0172f454baf2bb44a85add182 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.435572 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 443ae0978f2145658d910e2ad9fa38d0 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.435806 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73407db0172f454baf2bb44a85add182. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.435818 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 443ae0978f2145658d910e2ad9fa38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.435773 13140 tablet_service.cc:1467] Processing CreateTablet for tablet 443ae0978f2145658d910e2ad9fa38d0 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.436002 13140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 443ae0978f2145658d910e2ad9fa38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.436043 13271 tablet_service.cc:1467] Processing CreateTablet for tablet 56314112ecb94098950fb77e1dd233aa (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.436267 13271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56314112ecb94098950fb77e1dd233aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.437767 14348 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:44.437773 14352 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:44.438127 13139 tablet_service.cc:1467] Processing CreateTablet for tablet 56314112ecb94098950fb77e1dd233aa (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.438341 13139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56314112ecb94098950fb77e1dd233aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.438557 14348 tablet_bootstrap.cc:654] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.438577 14352 tablet_bootstrap.cc:654] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.439497 14346 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:44.440426 14346 tablet_bootstrap.cc:654] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.441330 12875 tablet_service.cc:1467] Processing CreateTablet for tablet 443ae0978f2145658d910e2ad9fa38d0 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.441582 12875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 443ae0978f2145658d910e2ad9fa38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.441704 14346 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:44.441785 14346 ts_tablet_manager.cc:1367] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.441926 14351 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:44.442204 14346 raft_consensus.cc:348] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.442291 14346 raft_consensus.cc:374] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.442318 14346 raft_consensus.cc:729] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.442428 14346 consensus_queue.cc:260] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.442674 14346 ts_tablet_manager.cc:1398] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.442677 14351 tablet_bootstrap.cc:654] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.442822 14346 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:44.443451 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 56314112ecb94098950fb77e1dd233aa (DEFAULT_TABLE table=rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.443575 14346 tablet_bootstrap.cc:654] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.443665 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56314112ecb94098950fb77e1dd233aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.444595 14346 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:44.444669 14346 ts_tablet_manager.cc:1367] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.445035 14346 raft_consensus.cc:348] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.445137 14346 raft_consensus.cc:374] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.445165 14346 raft_consensus.cc:729] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.445286 14346 consensus_queue.cc:260] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.445325 14352 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:44.445405 14352 ts_tablet_manager.cc:1367] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240501 14:09:44.445549 14346 ts_tablet_manager.cc:1398] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.445691 14346 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:44.445843 14352 raft_consensus.cc:348] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.445950 14352 raft_consensus.cc:374] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.445986 14352 raft_consensus.cc:729] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.446099 14352 consensus_queue.cc:260] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.446156 14351 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:44.446225 14351 ts_tablet_manager.cc:1367] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:09:44.446367 14352 ts_tablet_manager.cc:1398] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:44.446468 14346 tablet_bootstrap.cc:654] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.446517 14352 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:44.446616 14351 raft_consensus.cc:348] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.446713 14351 raft_consensus.cc:374] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.446748 14351 raft_consensus.cc:729] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.446867 14351 consensus_queue.cc:260] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.447131 14351 ts_tablet_manager.cc:1398] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.447284 14351 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:44.447351 14352 tablet_bootstrap.cc:654] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.448019 14346 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:44.448063 14351 tablet_bootstrap.cc:654] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.448132 14346 ts_tablet_manager.cc:1367] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:44.448647 14352 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:44.448710 14352 ts_tablet_manager.cc:1367] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:09:44.448688 14346 raft_consensus.cc:348] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.448771 14346 raft_consensus.cc:374] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.448802 14346 raft_consensus.cc:729] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.448925 14346 consensus_queue.cc:260] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.449213 14346 ts_tablet_manager.cc:1398] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.449301 14352 raft_consensus.cc:348] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.449512 14352 raft_consensus.cc:374] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.449549 14352 raft_consensus.cc:729] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.450152 14351 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:44.450227 14351 ts_tablet_manager.cc:1367] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:09:44.450335 14348 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:44.450415 14348 ts_tablet_manager.cc:1367] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240501 14:09:44.450574 14352 consensus_queue.cc:260] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.450628 14351 raft_consensus.cc:348] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.450716 14351 raft_consensus.cc:374] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.450745 14351 raft_consensus.cc:729] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.450811 14348 raft_consensus.cc:348] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.450843 14351 consensus_queue.cc:260] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.450899 14352 ts_tablet_manager.cc:1398] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:09:44.450898 14348 raft_consensus.cc:374] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.450930 14348 raft_consensus.cc:729] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.451036 14348 consensus_queue.cc:260] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.451105 14351 ts_tablet_manager.cc:1398] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:44.451273 14348 ts_tablet_manager.cc:1398] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.451961 14348 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:44.452960 14348 tablet_bootstrap.cc:654] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.454154 14348 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:44.454234 14348 ts_tablet_manager.cc:1367] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:44.454627 14348 raft_consensus.cc:348] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.454725 14348 raft_consensus.cc:374] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.454754 14348 raft_consensus.cc:729] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.454869 14348 consensus_queue.cc:260] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.455189 14348 ts_tablet_manager.cc:1398] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.593550 14343 raft_consensus.cc:482] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:44.593679 14343 raft_consensus.cc:504] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.594094 14343 leader_election.cc:288] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:44.594404 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "443ae0978f2145658d910e2ad9fa38d0" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:44.594429 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "443ae0978f2145658d910e2ad9fa38d0" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:44.594522 13291 raft_consensus.cc:2454] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:44.594547 12896 raft_consensus.cc:2454] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 0.
I20240501 14:09:44.594764 13097 leader_election.cc:302] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:44.594916 14343 raft_consensus.cc:2781] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:44.594974 14343 raft_consensus.cc:482] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:44.595008 14343 raft_consensus.cc:3037] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.595791 14343 raft_consensus.cc:504] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.596071 14343 leader_election.cc:288] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Requested vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:44.596349 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "443ae0978f2145658d910e2ad9fa38d0" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:44.596365 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "443ae0978f2145658d910e2ad9fa38d0" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:44.596434 12896 raft_consensus.cc:3037] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.596462 13291 raft_consensus.cc:3037] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.597088 12896 raft_consensus.cc:2454] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:44.597127 13291 raft_consensus.cc:2454] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 1.
I20240501 14:09:44.597332 13094 leader_election.cc:302] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:44.597472 14343 raft_consensus.cc:2781] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:44.597620 14343 raft_consensus.cc:686] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:44.597765 14343 consensus_queue.cc:237] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } }
I20240501 14:09:44.598691 14321 raft_consensus.cc:482] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:44.598775 14321 raft_consensus.cc:504] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.599094 14321 leader_election.cc:288] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:44.599375 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73407db0172f454baf2bb44a85add182" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:44.599427 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73407db0172f454baf2bb44a85add182" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:44.599483 12896 raft_consensus.cc:2454] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:44.599525 13160 raft_consensus.cc:2454] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:44.599748 12963 leader_election.cc:302] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:44.599951 14321 raft_consensus.cc:2781] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:44.599997 14321 raft_consensus.cc:482] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:44.600041 14321 raft_consensus.cc:3037] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.600224 12748 catalog_manager.cc:5484] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:44.600729 14321 raft_consensus.cc:504] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.601058 14321 leader_election.cc:288] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:44.601305 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73407db0172f454baf2bb44a85add182" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:44.601393 12896 raft_consensus.cc:3037] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.601523 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73407db0172f454baf2bb44a85add182" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:44.601632 13160 raft_consensus.cc:3037] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.602092 12896 raft_consensus.cc:2454] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:44.602340 12962 leader_election.cc:302] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:44.602401 13160 raft_consensus.cc:2454] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:44.602628 14321 raft_consensus.cc:2781] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:44.602716 14321 raft_consensus.cc:686] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Becoming Leader. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Running, Role: LEADER
I20240501 14:09:44.602836 14321 consensus_queue.cc:237] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.604276 12748 catalog_manager.cc:5484] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } }
I20240501 14:09:44.664431 14321 raft_consensus.cc:482] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:44.664551 14321 raft_consensus.cc:504] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.664921 14321 leader_election.cc:288] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:44.665282 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56314112ecb94098950fb77e1dd233aa" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:44.665280 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56314112ecb94098950fb77e1dd233aa" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:44.665406 13291 raft_consensus.cc:2454] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:44.665411 13160 raft_consensus.cc:2454] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:44.665663 12965 leader_election.cc:302] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:44.665823 14321 raft_consensus.cc:2781] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:44.665881 14321 raft_consensus.cc:482] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:44.665918 14321 raft_consensus.cc:3037] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.666668 14321 raft_consensus.cc:504] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.666965 14321 leader_election.cc:288] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Requested vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:44.667263 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56314112ecb94098950fb77e1dd233aa" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:44.667279 13291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56314112ecb94098950fb77e1dd233aa" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:44.667379 13160 raft_consensus.cc:3037] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.667404 13291 raft_consensus.cc:3037] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.668061 13160 raft_consensus.cc:2454] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:44.668100 13291 raft_consensus.cc:2454] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:44.668323 12963 leader_election.cc:302] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:44.668480 14321 raft_consensus.cc:2781] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:44.668560 14321 raft_consensus.cc:686] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Becoming Leader. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Running, Role: LEADER
I20240501 14:09:44.668689 14321 consensus_queue.cc:237] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.670353 12748 catalog_manager.cc:5484] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:44.785986 14415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:44.786193 14415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:44.789144 14415 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240501 14:09:44.861593 13291 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.862234 14269 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.862552 12896 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.863186 13161 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.863690 13160 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.863871 14321 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.864135 14321 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.865427 13292 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.866044 14343 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.867259 12896 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:44.872112 14321 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:44.876243 14393 mvcc.cc:205] Tried to move back new op lower bound from 7022889307588259840 to 7022889306534383616. Current Snapshot: MvccSnapshot[applied={T|T < 7022889307588259840}]
I20240501 14:09:44.885072 14411 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 100.995 ms
time per row: 0.0504975 ms
I20240501 14:09:44.928336 12752 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:44.933434 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc05e3adc8e4ed0b45d275725f289f4 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.933739 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc05e3adc8e4ed0b45d275725f289f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.934478 13139 tablet_service.cc:1467] Processing CreateTablet for tablet 2f58cd9e96244a2da7954bdaf9272b82 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.934724 13139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f58cd9e96244a2da7954bdaf9272b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.935804 14346 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:44.936580 14346 tablet_bootstrap.cc:654] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.936726 13009 tablet_service.cc:1467] Processing CreateTablet for tablet 68927b876c634175841f0859e3f535f4 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.937000 13009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68927b876c634175841f0859e3f535f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.938526 14348 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:44.939076 12875 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc05e3adc8e4ed0b45d275725f289f4 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.939101 12877 tablet_service.cc:1467] Processing CreateTablet for tablet 2f58cd9e96244a2da7954bdaf9272b82 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.939294 14348 tablet_bootstrap.cc:654] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.939101 12876 tablet_service.cc:1467] Processing CreateTablet for tablet 68927b876c634175841f0859e3f535f4 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.939568 13271 tablet_service.cc:1467] Processing CreateTablet for tablet 68927b876c634175841f0859e3f535f4 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.939373 12875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc05e3adc8e4ed0b45d275725f289f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.936726 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc05e3adc8e4ed0b45d275725f289f4 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.939810 13271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68927b876c634175841f0859e3f535f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.939901 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc05e3adc8e4ed0b45d275725f289f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.941649 12876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68927b876c634175841f0859e3f535f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.941756 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 2f58cd9e96244a2da7954bdaf9272b82 (DEFAULT_TABLE table=rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:44.941939 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f58cd9e96244a2da7954bdaf9272b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.942822 14352 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:44.943436 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f58cd9e96244a2da7954bdaf9272b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:44.943555 14352 tablet_bootstrap.cc:654] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.944115 14351 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:44.944535 14352 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:44.944600 14352 ts_tablet_manager.cc:1367] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.944876 14351 tablet_bootstrap.cc:654] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.945005 14352 raft_consensus.cc:348] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.945094 14352 raft_consensus.cc:374] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.945123 14352 raft_consensus.cc:729] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.945235 14352 consensus_queue.cc:260] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.945487 14352 ts_tablet_manager.cc:1398] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:44.945662 14352 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:44.946374 14351 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:44.946450 14351 ts_tablet_manager.cc:1367] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:44.946524 14348 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:44.946594 14348 ts_tablet_manager.cc:1367] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240501 14:09:44.946779 14352 tablet_bootstrap.cc:654] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.946846 14351 raft_consensus.cc:348] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.946933 14351 raft_consensus.cc:374] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.946961 14351 raft_consensus.cc:729] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.947001 14348 raft_consensus.cc:348] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.947096 14348 raft_consensus.cc:374] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.947074 14351 consensus_queue.cc:260] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.947129 14348 raft_consensus.cc:729] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.947232 14348 consensus_queue.cc:260] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.947371 14351 ts_tablet_manager.cc:1398] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.947521 14351 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:44.947512 14348 ts_tablet_manager.cc:1398] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:44.947667 14348 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:44.947845 14352 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:44.947911 14352 ts_tablet_manager.cc:1367] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 14:09:44.948257 14351 tablet_bootstrap.cc:654] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.948299 14352 raft_consensus.cc:348] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.948379 14352 raft_consensus.cc:374] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.948406 14352 raft_consensus.cc:729] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.948467 14348 tablet_bootstrap.cc:654] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.948510 14352 consensus_queue.cc:260] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.948758 14352 ts_tablet_manager.cc:1398] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:44.949450 14351 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:44.949525 14351 ts_tablet_manager.cc:1367] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:44.949739 14346 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:44.949831 14346 ts_tablet_manager.cc:1367] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240501 14:09:44.949918 14351 raft_consensus.cc:348] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.950004 14351 raft_consensus.cc:374] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.950037 14351 raft_consensus.cc:729] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.950145 14351 consensus_queue.cc:260] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.950253 14346 raft_consensus.cc:348] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.950353 14346 raft_consensus.cc:374] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.950385 14346 raft_consensus.cc:729] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.950384 14351 ts_tablet_manager.cc:1398] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:44.950484 14346 consensus_queue.cc:260] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.950768 14346 ts_tablet_manager.cc:1398] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:44.950950 14346 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:44.950965 14351 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:44.951979 14351 tablet_bootstrap.cc:654] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.952095 14346 tablet_bootstrap.cc:654] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:44.952906 14348 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:44.952982 14348 ts_tablet_manager.cc:1367] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240501 14:09:44.953817 14348 raft_consensus.cc:348] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.953907 14348 raft_consensus.cc:374] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.953943 14348 raft_consensus.cc:729] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.954257 14348 consensus_queue.cc:260] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:44.954336 14346 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:44.954397 14346 ts_tablet_manager.cc:1367] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 14:09:44.954543 14348 ts_tablet_manager.cc:1398] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:09:44.955132 14346 raft_consensus.cc:348] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.955214 14351 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:44.955231 14346 raft_consensus.cc:374] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.955268 14346 raft_consensus.cc:729] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.955281 14351 ts_tablet_manager.cc:1367] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 14:09:44.955390 14346 consensus_queue.cc:260] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.955621 14346 ts_tablet_manager.cc:1398] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:44.955763 14351 raft_consensus.cc:348] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.955839 14351 raft_consensus.cc:374] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:44.955868 14351 raft_consensus.cc:729] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:44.955971 14351 consensus_queue.cc:260] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.956193 14351 ts_tablet_manager.cc:1398] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:44.979853 14450 raft_consensus.cc:482] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:44.979950 14450 raft_consensus.cc:504] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.980286 14450 leader_election.cc:288] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:44.980556 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68927b876c634175841f0859e3f535f4" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" is_pre_election: true
I20240501 14:09:44.980590 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68927b876c634175841f0859e3f535f4" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:44.980666 12897 raft_consensus.cc:2454] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:44.980722 13293 raft_consensus.cc:2454] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 0.
I20240501 14:09:44.980935 12962 leader_election.cc:302] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:44.981087 14450 raft_consensus.cc:2781] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:44.981142 14450 raft_consensus.cc:482] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:44.981175 14450 raft_consensus.cc:3037] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.981954 14450 raft_consensus.cc:504] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.982300 14450 leader_election.cc:288] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:44.982674 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68927b876c634175841f0859e3f535f4" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:44.982791 13293 raft_consensus.cc:3037] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.983587 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68927b876c634175841f0859e3f535f4" candidate_uuid: "3021737e50984ca4bb2ad151dc4776ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:44.983660 13293 raft_consensus.cc:2454] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:44.983700 12897 raft_consensus.cc:3037] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:44.983897 12965 leader_election.cc:302] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:44.984040 14450 raft_consensus.cc:2781] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:44.984120 14450 raft_consensus.cc:686] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Becoming Leader. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Running, Role: LEADER
I20240501 14:09:44.984246 14450 consensus_queue.cc:237] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:44.984525 12897 raft_consensus.cc:2454] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3021737e50984ca4bb2ad151dc4776ff in term 1.
I20240501 14:09:44.985742 12748 catalog_manager.cc:5484] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:45.004524 13899 raft_consensus.cc:482] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:45.004609 13899 raft_consensus.cc:504] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:45.004882 13899 leader_election.cc:288] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:45.005190 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc05e3adc8e4ed0b45d275725f289f4" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:45.005249 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc05e3adc8e4ed0b45d275725f289f4" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:45.005324 13160 raft_consensus.cc:2454] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:45.005352 13028 raft_consensus.cc:2454] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:45.005590 12832 leader_election.cc:302] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:45.005775 13899 raft_consensus.cc:2781] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:45.005838 13899 raft_consensus.cc:482] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:45.005889 13899 raft_consensus.cc:3037] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.006556 13899 raft_consensus.cc:504] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:45.006868 13899 leader_election.cc:288] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Requested vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:45.007154 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc05e3adc8e4ed0b45d275725f289f4" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:45.007150 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc05e3adc8e4ed0b45d275725f289f4" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:45.007247 13160 raft_consensus.cc:3037] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.007248 13028 raft_consensus.cc:3037] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.007920 13160 raft_consensus.cc:2454] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:45.007967 13028 raft_consensus.cc:2454] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:45.008217 12832 leader_election.cc:302] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:45.008369 13899 raft_consensus.cc:2781] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:45.008446 13899 raft_consensus.cc:686] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:45.008569 13899 consensus_queue.cc:237] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } }
I20240501 14:09:45.010063 12748 catalog_manager.cc:5484] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:45.011651 13899 raft_consensus.cc:482] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:45.011734 13899 raft_consensus.cc:504] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.012049 13899 leader_election.cc:288] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:45.012341 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:45.012372 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e" is_pre_election: true
I20240501 14:09:45.012430 13160 raft_consensus.cc:2454] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:45.012477 13293 raft_consensus.cc:2454] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:45.012657 12831 leader_election.cc:302] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:45.012845 13899 raft_consensus.cc:2781] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:45.012888 13899 raft_consensus.cc:482] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:45.012921 13899 raft_consensus.cc:3037] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.013669 13899 raft_consensus.cc:504] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.013933 13899 leader_election.cc:288] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Requested vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:45.014223 13293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:45.014318 13293 raft_consensus.cc:3037] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.014945 13293 raft_consensus.cc:2454] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:45.015125 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:45.015172 12833 leader_election.cc:302] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:45.015211 13160 raft_consensus.cc:3037] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.015314 13899 raft_consensus.cc:2781] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:45.015403 13899 raft_consensus.cc:686] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:45.015513 13899 consensus_queue.cc:237] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.016011 13160 raft_consensus.cc:2454] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:45.016911 12748 catalog_manager.cc:5484] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:45.123773 14467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:45.123968 14467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:45.126837 14467 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240501 14:09:45.197736 13160 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.198158 13292 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.198129 14361 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.198678 13028 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.198719 12897 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.199002 13899 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.199021 14450 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.199308 14450 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.203866 13292 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:09:45.203897 13161 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:09:45.204406 13899 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.204519 14361 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 93.4862 ms
time per row: 0.0467431 ms
I20240501 14:09:45.258527 12752 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:09:45.263473 13139 tablet_service.cc:1467] Processing CreateTablet for tablet 15661c749d7a4cd688d4d666a5eca1c8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.263564 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 15661c749d7a4cd688d4d666a5eca1c8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.263761 13139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15661c749d7a4cd688d4d666a5eca1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.263824 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15661c749d7a4cd688d4d666a5eca1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.264417 12876 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.264645 12876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.265491 14346 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:45.265838 14348 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:45.266201 14346 tablet_bootstrap.cc:654] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.266525 14351 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7: Bootstrap starting.
I20240501 14:09:45.266644 14348 tablet_bootstrap.cc:654] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.267216 14351 tablet_bootstrap.cc:654] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.267660 14348 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:45.267609 13273 tablet_service.cc:1467] Processing CreateTablet for tablet 15661c749d7a4cd688d4d666a5eca1c8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.267740 14348 ts_tablet_manager.cc:1367] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:45.267868 13273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15661c749d7a4cd688d4d666a5eca1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.268165 14351 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7: No bootstrap required, opened a new log
I20240501 14:09:45.268142 14348 raft_consensus.cc:348] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.268239 14351 ts_tablet_manager.cc:1367] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:09:45.268242 14348 raft_consensus.cc:374] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.268275 14348 raft_consensus.cc:729] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.268364 14348 consensus_queue.cc:260] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.268605 14348 ts_tablet_manager.cc:1398] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:45.268608 14351 raft_consensus.cc:348] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.268702 14351 raft_consensus.cc:374] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.268729 14351 raft_consensus.cc:729] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.268836 14351 consensus_queue.cc:260] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.269063 14351 ts_tablet_manager.cc:1398] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:45.269949 14352 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:45.270025 13271 tablet_service.cc:1467] Processing CreateTablet for tablet 2a19716d4a9b4acd8a3515b48d9eac32 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.270233 13271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a19716d4a9b4acd8a3515b48d9eac32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.270850 14352 tablet_bootstrap.cc:654] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.271872 14352 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:45.271950 14352 ts_tablet_manager.cc:1367] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 14:09:45.271934 13139 tablet_service.cc:1467] Processing CreateTablet for tablet 2a19716d4a9b4acd8a3515b48d9eac32 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.272140 13139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a19716d4a9b4acd8a3515b48d9eac32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.272362 14352 raft_consensus.cc:348] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.272455 14352 raft_consensus.cc:374] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.272432 13009 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.272485 14352 raft_consensus.cc:729] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.272626 13009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.272608 14352 consensus_queue.cc:260] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.272854 14352 ts_tablet_manager.cc:1398] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:45.272967 14346 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:45.273000 14352 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: Bootstrap starting.
I20240501 14:09:45.273039 14346 ts_tablet_manager.cc:1367] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 14:09:45.273440 14346 raft_consensus.cc:348] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.273543 14346 raft_consensus.cc:374] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.273574 14346 raft_consensus.cc:729] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.273664 14346 consensus_queue.cc:260] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.273808 14352 tablet_bootstrap.cc:654] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.273921 14346 ts_tablet_manager.cc:1398] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:45.274001 13141 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.274073 14346 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:45.274214 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.274883 14346 tablet_bootstrap.cc:654] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.275590 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 2a19716d4a9b4acd8a3515b48d9eac32 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:09:45.275821 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a19716d4a9b4acd8a3515b48d9eac32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:45.277944 14348 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:45.277979 14346 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:45.278046 14346 ts_tablet_manager.cc:1367] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 14:09:45.278429 14346 raft_consensus.cc:348] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.278528 14346 raft_consensus.cc:374] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.278577 14346 raft_consensus.cc:729] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.278690 14346 consensus_queue.cc:260] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.278761 14348 tablet_bootstrap.cc:654] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.278909 14346 ts_tablet_manager.cc:1398] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:45.279049 14346 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739: Bootstrap starting.
I20240501 14:09:45.279801 14346 tablet_bootstrap.cc:654] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.280176 14348 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:45.280297 14348 ts_tablet_manager.cc:1367] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:45.280884 14352 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: No bootstrap required, opened a new log
I20240501 14:09:45.280961 14352 ts_tablet_manager.cc:1367] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240501 14:09:45.281340 14352 raft_consensus.cc:348] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.281334 14348 raft_consensus.cc:348] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.281424 14352 raft_consensus.cc:374] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.281424 14348 raft_consensus.cc:374] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.281453 14352 raft_consensus.cc:729] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.281459 14348 raft_consensus.cc:729] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.281558 14352 consensus_queue.cc:260] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.281574 14348 consensus_queue.cc:260] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.281810 14352 ts_tablet_manager.cc:1398] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:45.281869 14348 ts_tablet_manager.cc:1398] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:45.282507 14348 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: Bootstrap starting.
I20240501 14:09:45.282943 14346 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739: No bootstrap required, opened a new log
I20240501 14:09:45.283013 14346 ts_tablet_manager.cc:1367] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 14:09:45.283296 14348 tablet_bootstrap.cc:654] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: Neither blocks nor log segments found. Creating new log.
I20240501 14:09:45.283388 14346 raft_consensus.cc:348] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.283473 14346 raft_consensus.cc:374] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.283505 14346 raft_consensus.cc:729] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.283598 14346 consensus_queue.cc:260] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.283883 14346 ts_tablet_manager.cc:1398] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:45.284332 14348 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: No bootstrap required, opened a new log
I20240501 14:09:45.284418 14348 ts_tablet_manager.cc:1367] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:09:45.284821 14348 raft_consensus.cc:348] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.284911 14348 raft_consensus.cc:374] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:09:45.285086 14348 raft_consensus.cc:729] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Initialized, Role: FOLLOWER
I20240501 14:09:45.285226 14348 consensus_queue.cc:260] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.285535 14348 ts_tablet_manager.cc:1398] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:09:45.375694 14463 raft_consensus.cc:482] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:45.375806 14463 raft_consensus.cc:504] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.376197 14463 leader_election.cc:288] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:45.376487 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a19716d4a9b4acd8a3515b48d9eac32" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:45.376519 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a19716d4a9b4acd8a3515b48d9eac32" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:45.376639 13161 raft_consensus.cc:2454] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:45.376639 13028 raft_consensus.cc:2454] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:45.376897 13226 leader_election.cc:302] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879199fbf918416cb23b0376dde0aa6e, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:45.377061 14463 raft_consensus.cc:2781] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:45.377122 14463 raft_consensus.cc:482] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:45.377166 14463 raft_consensus.cc:3037] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.377945 14463 raft_consensus.cc:504] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.378206 14463 leader_election.cc:288] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:45.378495 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a19716d4a9b4acd8a3515b48d9eac32" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:45.378520 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a19716d4a9b4acd8a3515b48d9eac32" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:45.378604 13161 raft_consensus.cc:3037] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.378604 13028 raft_consensus.cc:3037] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.379479 13161 raft_consensus.cc:2454] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:45.379479 13028 raft_consensus.cc:2454] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:45.379765 13227 leader_election.cc:302] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:45.379904 14463 raft_consensus.cc:2781] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:45.379984 14463 raft_consensus.cc:686] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Leader. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:09:45.380108 14463 consensus_queue.cc:237] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.381784 12748 catalog_manager.cc:5484] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 879199fbf918416cb23b0376dde0aa6e (127.6.153.196). New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } }
I20240501 14:09:45.487809 14361 raft_consensus.cc:482] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:45.487917 14361 raft_consensus.cc:504] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.488294 14361 leader_election.cc:288] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:45.488584 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2d06a2ae3f4fe5a515d142d3e608e8" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:45.488611 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2d06a2ae3f4fe5a515d142d3e608e8" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:45.488708 13028 raft_consensus.cc:2454] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:45.488739 13161 raft_consensus.cc:2454] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 0.
I20240501 14:09:45.488963 12832 leader_election.cc:302] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:45.489122 14361 raft_consensus.cc:2781] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:45.489187 14361 raft_consensus.cc:482] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:45.489223 14361 raft_consensus.cc:3037] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.490192 14361 raft_consensus.cc:504] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.490581 14361 leader_election.cc:288] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:45.490888 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2d06a2ae3f4fe5a515d142d3e608e8" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:45.491001 13028 raft_consensus.cc:3037] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.491151 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2d06a2ae3f4fe5a515d142d3e608e8" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:45.491272 13161 raft_consensus.cc:3037] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.491899 13028 raft_consensus.cc:2454] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:45.492177 13161 raft_consensus.cc:2454] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 1.
I20240501 14:09:45.492225 12832 leader_election.cc:302] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:45.492365 14361 raft_consensus.cc:2781] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:45.492444 14361 raft_consensus.cc:686] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:45.492727 14361 consensus_queue.cc:237] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } }
I20240501 14:09:45.494454 14463 raft_consensus.cc:482] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:09:45.494545 14463 raft_consensus.cc:504] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.494557 12748 catalog_manager.cc:5484] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:45.494885 14463 leader_election.cc:288] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:45.495183 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff" is_pre_election: true
I20240501 14:09:45.495183 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739" is_pre_election: true
I20240501 14:09:45.495293 13028 raft_consensus.cc:2454] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:45.495293 13161 raft_consensus.cc:2454] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 0.
I20240501 14:09:45.495559 13227 leader_election.cc:302] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:45.495985 14463 raft_consensus.cc:2781] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:09:45.496086 14463 raft_consensus.cc:482] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:09:45.496137 14463 raft_consensus.cc:3037] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.496908 14463 raft_consensus.cc:504] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.497251 14463 leader_election.cc:288] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Requested vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:45.497530 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:45.497530 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8" candidate_uuid: "879199fbf918416cb23b0376dde0aa6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:45.497633 13028 raft_consensus.cc:3037] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.497633 13161 raft_consensus.cc:3037] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:09:45.498418 13028 raft_consensus.cc:2454] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:45.498418 13161 raft_consensus.cc:2454] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879199fbf918416cb23b0376dde0aa6e in term 1.
I20240501 14:09:45.498658 13227 leader_election.cc:302] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:45.498800 14463 raft_consensus.cc:2781] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:09:45.498898 14463 raft_consensus.cc:686] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Leader. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:09:45.499020 14463 consensus_queue.cc:237] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } }
I20240501 14:09:45.500502 12748 catalog_manager.cc:5484] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 879199fbf918416cb23b0376dde0aa6e (127.6.153.196). New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } }
W20240501 14:09:45.662061 14520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:45.662287 14520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:45.665485 14520 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240501 14:09:45.772436 13161 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.772845 14361 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.773123 13028 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.773653 13028 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.773650 13160 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.774008 14463 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.774806 14463 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.776213 14361 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.778208 13159 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.778519 13027 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:09:45.779072 14462 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:09:45.779747 14462 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 139.001 ms
time per row: 0.0695007 ms
I20240501 14:09:45.841714 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.197:37879
--local_ip_for_outbound_sockets=127.6.153.197
--tserver_master_addrs=127.6.153.254:41367
--webserver_port=39881
--webserver_interface=127.6.153.197
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 14:09:45.945906 14582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:45.946095 14582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:45.946158 14582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:45.949357 14582 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:45.949437 14582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:45.949533 14582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.197
I20240501 14:09:45.953054 14582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.197:37879
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.6.153.197
--webserver_port=39881
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.14582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.197
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:45.953778 14582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:45.954763 14582 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:45.962389 14588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:45.962815 14587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:45.963156 14590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:45.963606 14582 server_base.cc:1034] running on GCE node
I20240501 14:09:45.963933 14582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:45.964430 14582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:45.965571 14582 hybrid_clock.cc:648] HybridClock initialized: now 1714572585965558 us; error 29 us; skew 500 ppm
I20240501 14:09:45.967296 14582 webserver.cc:460] Webserver started at http://127.6.153.197:39881/ using document root <none> and password file <none>
I20240501 14:09:45.967818 14582 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:45.967898 14582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:45.971347 14582 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20240501 14:09:45.973126 14596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:45.974036 14582 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:45.974123 14582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "2986bfcc5da34cbca2cad1643b6e35c9"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:45.974506 14582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:45.994014 14582 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:45.994583 14582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:45.994735 14582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:45.995258 14582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:45.996076 14582 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:45.996127 14582 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:45.996163 14582 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:45.996183 14582 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.003692 14582 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.197:37879
I20240501 14:09:46.003723 14710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.197:37879 every 8 connection(s)
I20240501 14:09:46.004707 14582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 14:09:46.007148 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 14582
I20240501 14:09:46.007586 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.198:38509
--local_ip_for_outbound_sockets=127.6.153.198
--tserver_master_addrs=127.6.153.254:41367
--webserver_port=45069
--webserver_interface=127.6.153.198
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:46.013700 14711 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:46.013883 14711 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:46.014334 14711 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:46.015352 12745 ts_manager.cc:194] Re-registered known tserver with Master: 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879)
I20240501 14:09:46.015910 12745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.197:39875
W20240501 14:09:46.110358 14714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:46.110541 14714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:46.110599 14714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:46.113539 14714 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:46.113600 14714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:46.113679 14714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.198
I20240501 14:09:46.117144 14714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.198:38509
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.6.153.198
--webserver_port=45069
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.14714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.198
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:46.118002 14714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:46.119019 14714 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:46.126911 14719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:46.127235 14722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:46.127628 14720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:46.128044 14714 server_base.cc:1034] running on GCE node
I20240501 14:09:46.128376 14714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:46.128866 14714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:46.130007 14714 hybrid_clock.cc:648] HybridClock initialized: now 1714572586129990 us; error 28 us; skew 500 ppm
I20240501 14:09:46.131697 14714 webserver.cc:460] Webserver started at http://127.6.153.198:45069/ using document root <none> and password file <none>
I20240501 14:09:46.132212 14714 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:46.132282 14714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:46.135412 14714 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:09:46.137125 14728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.138008 14714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:46.138100 14714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "18bef23ddc624bd784ea38fc6ef00040"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:46.138463 14714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:46.163502 14714 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:46.164062 14714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:46.164199 14714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:46.164685 14714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:46.165490 14714 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:46.165537 14714 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.165586 14714 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:46.165611 14714 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.173621 14714 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.198:38509
I20240501 14:09:46.173676 14842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.198:38509 every 8 connection(s)
I20240501 14:09:46.174536 14714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 14:09:46.182710 14843 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:46.182778 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 14714
I20240501 14:09:46.182919 14843 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:46.183212 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.199:38347
--local_ip_for_outbound_sockets=127.6.153.199
--tserver_master_addrs=127.6.153.254:41367
--webserver_port=42567
--webserver_interface=127.6.153.199
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:46.183507 14843 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:46.184515 12745 ts_manager.cc:194] Re-registered known tserver with Master: 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509)
I20240501 14:09:46.185034 12745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.198:49105
W20240501 14:09:46.288969 14846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:46.289165 14846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:46.289207 14846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:46.292155 14846 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:46.292209 14846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:46.292287 14846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.199
I20240501 14:09:46.295814 14846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.199:38347
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.6.153.199
--webserver_port=42567
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.14846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.199
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:46.296603 14846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:46.297638 14846 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:46.305465 14851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:46.305840 14852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:46.306310 14854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:46.306607 14846 server_base.cc:1034] running on GCE node
I20240501 14:09:46.306993 14846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:46.307543 14846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:46.308682 14846 hybrid_clock.cc:648] HybridClock initialized: now 1714572586308670 us; error 24 us; skew 500 ppm
I20240501 14:09:46.310356 14846 webserver.cc:460] Webserver started at http://127.6.153.199:42567/ using document root <none> and password file <none>
I20240501 14:09:46.310843 14846 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:46.310907 14846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:46.314330 14846 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:46.316386 14860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.317389 14846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:46.317474 14846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "70c63e471e5b482b8b257c3e7543feb5"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:46.317854 14846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:46.351157 14846 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:46.351768 14846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:46.351898 14846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:46.352349 14846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:46.353147 14846 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:46.353196 14846 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.353233 14846 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:46.353253 14846 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.361086 14846 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.199:38347
I20240501 14:09:46.361146 14973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.199:38347 every 8 connection(s)
I20240501 14:09:46.362112 14846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 14:09:46.368924 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 14846
I20240501 14:09:46.369331 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.200:34743
--local_ip_for_outbound_sockets=127.6.153.200
--tserver_master_addrs=127.6.153.254:41367
--webserver_port=43281
--webserver_interface=127.6.153.200
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:46.370142 14974 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:46.370358 14974 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:46.370898 14974 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:46.371816 12745 ts_manager.cc:194] Re-registered known tserver with Master: 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347)
I20240501 14:09:46.372334 12745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.199:33341
W20240501 14:09:46.474957 14977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:46.475152 14977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:46.475203 14977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:46.478181 14977 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:46.478236 14977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:46.478322 14977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.200
I20240501 14:09:46.482039 14977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.200:34743
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.6.153.200
--webserver_port=43281
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.14977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.200
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:46.482823 14977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:46.483896 14977 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:09:46.491751 14983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:46.491987 14984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:46.492363 14986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:46.492838 14977 server_base.cc:1034] running on GCE node
I20240501 14:09:46.493228 14977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:46.493835 14977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:46.494988 14977 hybrid_clock.cc:648] HybridClock initialized: now 1714572586494972 us; error 30 us; skew 500 ppm
I20240501 14:09:46.496971 14977 webserver.cc:460] Webserver started at http://127.6.153.200:43281/ using document root <none> and password file <none>
I20240501 14:09:46.497467 14977 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:46.497531 14977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:46.500581 14977 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20240501 14:09:46.502393 14992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.503387 14977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:46.503489 14977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "3025ea3ec516466fbe5c5c4337733584"
format_stamp: "Formatted at 2024-05-01 14:09:36 on dist-test-slave-cmf0"
I20240501 14:09:46.503939 14977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:46.536773 14977 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:46.537315 14977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:46.537456 14977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:46.537990 14977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:46.538908 14977 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:46.538951 14977 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.538990 14977 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:46.539021 14977 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:46.548528 14977 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.200:34743
I20240501 14:09:46.548569 15105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.200:34743 every 8 connection(s)
I20240501 14:09:46.549458 14977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 14:09:46.554773 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 14977
I20240501 14:09:46.558142 15106 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:46.558359 15106 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:46.558928 15106 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:46.559834 12745 ts_manager.cc:194] Re-registered known tserver with Master: 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743)
I20240501 14:09:46.560468 12745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.200:46621
W20240501 14:09:46.658735 15110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:46.658937 15110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:46.661922 15110 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 14:09:46.782317 14644 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:46.786170 14766 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:46.786929 13271 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:09:46.789814 15040 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:46.798349 13007 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:46.818468 13141 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 14:09:46.821409 14908 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:46.833365 12876 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 14:09:46.834267 15040 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:46.835258 14644 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:46.840655 12876 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 14:09:46.842310 14766 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:46.845220 14908 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:46.847939 13007 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:46.860210 13271 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:09:46.865060 13141 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 14:09:47.002981 14766 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.009063 13007 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:47.011363 15040 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.017688 14711 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
I20240501 14:09:47.019387 14908 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.026669 14644 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.040606 13271 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:09:47.057600 12876 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
W20240501 14:09:47.057893 15226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:47.058149 15226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240501 14:09:47.058948 13141 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
W20240501 14:09:47.065853 15226 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 14:09:47.184960 14908 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.187425 14766 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.188555 14843 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
I20240501 14:09:47.197501 14644 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.200248 15040 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.207811 13271 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:09:47.208879 13007 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:47.210000 12876 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 14:09:47.215651 13141 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 14:09:47.240994 14766 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.250771 13271 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:09:47.251611 14908 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.257418 13007 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:47.257755 12876 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 14:09:47.260881 13141 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 14:09:47.281350 14644 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.294046 15040 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.335824 13159 consensus_queue.cc:237] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:47.337495 13028 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.337731 13292 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.337944 15356 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.338555 15356 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.340168 14343 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.341197 13292 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.343958 12750 catalog_manager.cc:5484] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.344641 13028 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
W20240501 14:09:47.347806 13097 consensus_peers.cc:485] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 439e22f89e944051b44d3263c43b2fc3. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.356519 13292 consensus_queue.cc:237] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:47.357882 13028 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.357957 13159 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.358269 14567 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.358636 15372 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.360785 13028 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.360810 13159 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.362529 15372 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.365890 12749 catalog_manager.cc:5484] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.375169 14974 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
W20240501 14:09:47.376245 13225 consensus_peers.cc:485] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78408b0cc50c438caf75876d8dcf88a3. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.378283 13159 consensus_queue.cc:237] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:47.379662 13292 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.380002 12897 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.380014 15355 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.380820 15355 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.382732 15355 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.383076 12897 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
W20240501 14:09:47.385094 13096 consensus_peers.cc:485] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e34fd488c214188aac9990af9de0db4. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.385269 12745 catalog_manager.cc:5484] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.385418 13292 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.394383 12897 consensus_queue.cc:237] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:47.395624 13292 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.395946 15214 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.396020 13028 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.396445 15214 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.397986 15392 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.399567 13028 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.401896 12749 catalog_manager.cc:5484] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:47.403517 12831 consensus_peers.cc:485] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe046c56dde144f9adbbbf779067476e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.404525 13292 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.411181 12897 consensus_queue.cc:237] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:47.412444 13159 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.412676 13292 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.412818 15392 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.413136 15392 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.414789 15391 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.416461 13292 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.417083 13159 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
W20240501 14:09:47.417966 12833 consensus_peers.cc:485] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37047d45ffe14570b28e459afdb66d7e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.418596 12750 catalog_manager.cc:5484] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.427632 12897 consensus_queue.cc:237] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:47.429013 13159 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.429406 13028 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.429756 15391 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.429945 15392 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.431768 15391 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.432152 13028 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
W20240501 14:09:47.434549 12830 consensus_peers.cc:485] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fc05e3adc8e4ed0b45d275725f289f4. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.434588 12745 catalog_manager.cc:5484] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.435019 13159 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.444306 13028 consensus_queue.cc:237] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:47.445823 12897 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.445900 13292 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.446214 15418 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.446384 15215 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.448168 15215 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.448554 12897 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.451691 12750 catalog_manager.cc:5484] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.452728 13292 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
W20240501 14:09:47.455487 12964 consensus_peers.cc:485] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 68927b876c634175841f0859e3f535f4. This is attempt 1: this message will repeat every 5th retry.
W20240501 14:09:47.462311 15352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:47.462561 15352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240501 14:09:47.464383 13028 consensus_queue.cc:237] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:47.465569 12897 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:09:47.465992 15352 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 14:09:47.465937 15215 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.466300 13292 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.466653 15215 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.468086 15215 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.468418 12897 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.470623 12749 catalog_manager.cc:5484] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.471733 13292 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
W20240501 14:09:47.477164 12965 consensus_peers.cc:485] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5053c0f554af4efaa221444af87dd9b9. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.482805 13292 consensus_queue.cc:237] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:47.484251 12897 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.484592 15372 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.484781 13159 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.485111 15372 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.487459 15373 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.487859 12897 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.487919 13159 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
W20240501 14:09:47.488027 13226 consensus_peers.cc:485] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d6516b6f77b4389aa53862668486bba. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.490484 12749 catalog_manager.cc:5484] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.497298 13159 consensus_queue.cc:237] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:47.498817 13028 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.499197 15361 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 14:09:47.499833 13096 consensus_peers.cc:485] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2affe37c7174ff1be13ba4807d1fffe. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.500162 12897 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.500552 15361 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.501304 15361 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.501649 13028 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.503818 12745 catalog_manager.cc:5484] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.506181 12897 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.513969 13159 consensus_queue.cc:237] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:47.515468 13028 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.515868 12897 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.516041 15355 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.516325 15360 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.517876 15360 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.518893 13028 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
W20240501 14:09:47.519500 13094 consensus_peers.cc:485] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecd37c11bf4c45de95f861c1eb4f20d6. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.521188 12750 catalog_manager.cc:5484] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.521350 12897 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.529858 13292 consensus_queue.cc:237] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:47.531302 13028 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.531664 12897 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.532068 14567 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.532389 14567 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.533950 14567 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
W20240501 14:09:47.535706 13228 consensus_peers.cc:485] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34f677061e42436ab08123227f2be7bf. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.536796 13028 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.537304 12897 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.539211 12745 catalog_manager.cc:5484] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.548404 13292 consensus_queue.cc:237] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:47.549870 13028 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.550572 13159 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.550915 15373 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.551092 15372 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 14:09:47.551513 13226 consensus_peers.cc:485] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15661c749d7a4cd688d4d666a5eca1c8. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.562170 15106 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
I20240501 14:09:47.562865 15373 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.564996 13159 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.565299 12745 catalog_manager.cc:5484] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.572821 13028 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.578204 12897 consensus_queue.cc:237] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:47.579722 13159 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.579967 13028 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.580410 15391 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.580777 15391 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.585350 15214 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
W20240501 14:09:47.585520 12832 consensus_peers.cc:485] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6e2d06a2ae3f4fe5a515d142d3e608e8. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.587179 13028 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.589305 12745 catalog_manager.cc:5484] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.592765 13159 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.597431 14766 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.597465 14644 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.605779 13292 consensus_queue.cc:237] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
W20240501 14:09:47.607285 13226 consensus_peers.cc:485] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5583f4e9fe9741649d365daa521355c8. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.607936 12897 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.608294 15372 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.610522 13028 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.610877 15372 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.612327 15484 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.614758 12897 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.614825 12748 catalog_manager.cc:5484] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.617288 13028 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.625363 12897 consensus_queue.cc:237] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:47.626629 13292 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:09:47.626907 12830 consensus_peers.cc:485] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 627fd32202d247c2af434ab8be152102. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.627048 15391 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.627790 13159 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.628161 15391 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.629135 15510 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.629542 13292 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.632427 13141 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 14:09:47.632489 12752 catalog_manager.cc:5484] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.634116 13159 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.634581 14908 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.636750 15040 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.636807 12876 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 14:09:47.639500 13007 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:47.647747 13271 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:09:47.667097 13292 consensus_queue.cc:237] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:47.671784 13028 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.672191 15484 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 14:09:47.672710 13228 consensus_peers.cc:485] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a19716d4a9b4acd8a3515b48d9eac32. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.673512 13159 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.673873 15484 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.675779 15372 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.676200 13159 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.677667 13028 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:47.679627 12752 catalog_manager.cc:5484] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.687975 13028 consensus_queue.cc:237] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:47.689522 13159 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.689882 15418 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.690060 13292 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
Runtime error: ksck discovered errors
W20240501 14:09:47.690611 12964 consensus_peers.cc:485] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56314112ecb94098950fb77e1dd233aa. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.690739 15419 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.692623 15215 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.692950 13292 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.693483 13159 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:47.695580 12745 catalog_manager.cc:5484] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.705621 13159 consensus_queue.cc:237] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:47.706936 12897 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.706935 13028 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.707408 15361 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.707592 15356 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.710086 15361 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
W20240501 14:09:47.710496 13095 consensus_peers.cc:485] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f63509c2039d49f09eee28a45e03ad81. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.712409 12752 catalog_manager.cc:5484] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:47.712618 13028 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.714010 12897 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:47.721473 13028 consensus_queue.cc:237] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:47.722951 13292 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.723428 12897 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:47.723459 15419 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.723789 15215 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:47.725629 15419 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.726625 12897 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.727103 13292 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:47.729074 12752 catalog_manager.cc:5484] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:47.730145 12962 consensus_peers.cc:485] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8ea7bbf6eec49728d69ec4b6e825e1e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:47.835053 15667 ts_tablet_manager.cc:902] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:47.836401 15667 tablet_copy_client.cc:308] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:47.843293 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 37047d45ffe14570b28e459afdb66d7e from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:41969)
I20240501 14:09:47.843451 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 37047d45ffe14570b28e459afdb66d7e from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:41969: session id = 70c63e471e5b482b8b257c3e7543feb5-37047d45ffe14570b28e459afdb66d7e
I20240501 14:09:47.844831 12917 tablet_copy_source_session.cc:205] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.845130 15679 ts_tablet_manager.cc:902] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:47.845767 15679 tablet_copy_client.cc:308] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:47.848961 15667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37047d45ffe14570b28e459afdb66d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:47.852140 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet 68927b876c634175841f0859e3f535f4 from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:42337)
I20240501 14:09:47.852272 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet 68927b876c634175841f0859e3f535f4 from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:42337: session id = 3025ea3ec516466fbe5c5c4337733584-68927b876c634175841f0859e3f535f4
I20240501 14:09:47.853556 13049 tablet_copy_source_session.cc:205] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.854800 15667 tablet_copy_client.cc:783] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.854972 15667 tablet_copy_client.cc:647] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
W20240501 14:09:47.856338 15562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:47.856566 15562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240501 14:09:47.856568 15669 tablet_copy_client.cc:1002] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.856765 15679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68927b876c634175841f0859e3f535f4. 1 dirs total, 0 dirs full, 0 dirs failed
W20240501 14:09:47.860464 15562 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 14:09:47.861933 15688 ts_tablet_manager.cc:902] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:47.862468 15688 tablet_copy_client.cc:308] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:47.862860 15679 tablet_copy_client.cc:783] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.863023 15679 tablet_copy_client.cc:647] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:47.865231 15681 tablet_copy_client.cc:1002] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.866117 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 439e22f89e944051b44d3263c43b2fc3 from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:52497)
I20240501 14:09:47.866264 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 439e22f89e944051b44d3263c43b2fc3 from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:52497: session id = 70c63e471e5b482b8b257c3e7543feb5-439e22f89e944051b44d3263c43b2fc3
I20240501 14:09:47.867405 13181 tablet_copy_source_session.cc:205] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.868361 15688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 439e22f89e944051b44d3263c43b2fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:47.869935 15688 tablet_copy_client.cc:783] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.870056 15688 tablet_copy_client.cc:647] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:47.871172 15692 tablet_copy_client.cc:1002] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.890456 15700 ts_tablet_manager.cc:902] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:47.891134 15700 tablet_copy_client.cc:308] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:47.894721 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet fe046c56dde144f9adbbbf779067476e from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:57299)
I20240501 14:09:47.894809 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet fe046c56dde144f9adbbbf779067476e from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:57299: session id = 2986bfcc5da34cbca2cad1643b6e35c9-fe046c56dde144f9adbbbf779067476e
I20240501 14:09:47.895574 12917 tablet_copy_source_session.cc:205] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.898345 15700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe046c56dde144f9adbbbf779067476e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:47.904191 15700 tablet_copy_client.cc:783] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.904475 15700 tablet_copy_client.cc:647] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:47.906461 15704 tablet_copy_client.cc:1002] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.919365 15727 ts_tablet_manager.cc:902] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:47.920204 15727 tablet_copy_client.cc:308] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:47.927035 15733 ts_tablet_manager.cc:902] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:47.927642 15734 ts_tablet_manager.cc:902] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:47.928135 15734 tablet_copy_client.cc:308] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:47.928503 15739 ts_tablet_manager.cc:902] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:47.929457 15739 tablet_copy_client.cc:308] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:47.931562 13313 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 78408b0cc50c438caf75876d8dcf88a3 from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:49463)
I20240501 14:09:47.931699 13313 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 78408b0cc50c438caf75876d8dcf88a3 from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:49463: session id = 18bef23ddc624bd784ea38fc6ef00040-78408b0cc50c438caf75876d8dcf88a3
I20240501 14:09:47.933017 13313 tablet_copy_source_session.cc:205] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.933507 13312 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 15661c749d7a4cd688d4d666a5eca1c8 from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:48871)
I20240501 14:09:47.933599 13312 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 15661c749d7a4cd688d4d666a5eca1c8 from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:48871: session id = 2986bfcc5da34cbca2cad1643b6e35c9-15661c749d7a4cd688d4d666a5eca1c8
I20240501 14:09:47.934266 13312 tablet_copy_source_session.cc:205] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.934551 15733 tablet_copy_client.cc:308] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:47.936441 15727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78408b0cc50c438caf75876d8dcf88a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:47.939649 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 7e34fd488c214188aac9990af9de0db4 from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:59777)
I20240501 14:09:47.939743 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 7e34fd488c214188aac9990af9de0db4 from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:59777: session id = 3025ea3ec516466fbe5c5c4337733584-7e34fd488c214188aac9990af9de0db4
I20240501 14:09:47.940630 13181 tablet_copy_source_session.cc:205] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.941953 15739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e34fd488c214188aac9990af9de0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:47.942831 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet ecd37c11bf4c45de95f861c1eb4f20d6 from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:59809)
I20240501 14:09:47.942925 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet ecd37c11bf4c45de95f861c1eb4f20d6 from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:59809: session id = 18bef23ddc624bd784ea38fc6ef00040-ecd37c11bf4c45de95f861c1eb4f20d6
I20240501 14:09:47.943661 13181 tablet_copy_source_session.cc:205] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.944367 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd37c11bf4c45de95f861c1eb4f20d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:47.944439 15739 tablet_copy_client.cc:783] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.944583 15739 tablet_copy_client.cc:647] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:47.945473 15740 tablet_copy_client.cc:1002] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.947782 15734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15661c749d7a4cd688d4d666a5eca1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:47.948218 15733 tablet_copy_client.cc:783] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.948385 15733 tablet_copy_client.cc:647] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:47.949492 15727 tablet_copy_client.cc:783] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.949517 15745 tablet_copy_client.cc:1002] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.949626 15727 tablet_copy_client.cc:647] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:47.949751 15734 tablet_copy_client.cc:783] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.949882 15734 tablet_copy_client.cc:647] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:47.950850 15738 tablet_copy_client.cc:1002] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.950896 15729 tablet_copy_client.cc:1002] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.957907 15748 ts_tablet_manager.cc:902] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:47.958482 15748 tablet_copy_client.cc:308] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:47.959932 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet c2affe37c7174ff1be13ba4807d1fffe from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:59777)
I20240501 14:09:47.960040 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet c2affe37c7174ff1be13ba4807d1fffe from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:59777: session id = 3025ea3ec516466fbe5c5c4337733584-c2affe37c7174ff1be13ba4807d1fffe
I20240501 14:09:47.960811 13181 tablet_copy_source_session.cc:205] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:47.962734 15748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2affe37c7174ff1be13ba4807d1fffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:47.964699 15748 tablet_copy_client.cc:783] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:47.964815 15748 tablet_copy_client.cc:647] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:47.965725 14766 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.968196 15752 tablet_copy_client.cc:1002] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:47.971375 15040 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.972512 14908 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.978647 14644 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:47.978721 12876 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 14:09:47.981596 13007 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:47.996390 13271 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:09:47.998575 15764 ts_tablet_manager.cc:902] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:47.999135 15764 tablet_copy_client.cc:308] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:47.999900 13141 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 14:09:48.015084 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:55411)
I20240501 14:09:48.015184 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:55411: session id = 3025ea3ec516466fbe5c5c4337733584-6e2d06a2ae3f4fe5a515d142d3e608e8
I20240501 14:09:48.016093 12917 tablet_copy_source_session.cc:205] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.023516 15764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.023804 15769 ts_tablet_manager.cc:902] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:48.024371 15769 tablet_copy_client.cc:308] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:48.026039 15764 tablet_copy_client.cc:783] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.026266 15764 tablet_copy_client.cc:647] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.027374 15765 tablet_copy_client.cc:1002] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.028331 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 4fc05e3adc8e4ed0b45d275725f289f4 from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:58259)
I20240501 14:09:48.028425 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 4fc05e3adc8e4ed0b45d275725f289f4 from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:58259: session id = 18bef23ddc624bd784ea38fc6ef00040-4fc05e3adc8e4ed0b45d275725f289f4
I20240501 14:09:48.029235 12917 tablet_copy_source_session.cc:205] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.029973 15769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc05e3adc8e4ed0b45d275725f289f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.030377 15774 ts_tablet_manager.cc:902] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:48.030869 15774 tablet_copy_client.cc:308] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:48.032034 15769 tablet_copy_client.cc:783] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.032183 15769 tablet_copy_client.cc:647] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.033031 15771 tablet_copy_client.cc:1002] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.033809 15779 ts_tablet_manager.cc:902] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:48.034255 15779 tablet_copy_client.cc:308] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:48.037997 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet 5053c0f554af4efaa221444af87dd9b9 from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:52399)
I20240501 14:09:48.038096 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet 5053c0f554af4efaa221444af87dd9b9 from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:52399: session id = 70c63e471e5b482b8b257c3e7543feb5-5053c0f554af4efaa221444af87dd9b9
I20240501 14:09:48.038534 13313 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 34f677061e42436ab08123227f2be7bf from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:41149)
I20240501 14:09:48.038630 13313 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 34f677061e42436ab08123227f2be7bf from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:41149: session id = 70c63e471e5b482b8b257c3e7543feb5-34f677061e42436ab08123227f2be7bf
I20240501 14:09:48.038935 13049 tablet_copy_source_session.cc:205] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.039515 13313 tablet_copy_source_session.cc:205] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.040308 15779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34f677061e42436ab08123227f2be7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.042161 15779 tablet_copy_client.cc:783] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.042305 15779 tablet_copy_client.cc:647] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.042629 15774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5053c0f554af4efaa221444af87dd9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.043601 15783 tablet_copy_client.cc:1002] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.044159 15785 ts_tablet_manager.cc:902] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:48.044428 15774 tablet_copy_client.cc:783] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.044562 15774 tablet_copy_client.cc:647] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.044736 15785 tablet_copy_client.cc:308] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
Runtime error: ksck discovered errors
I20240501 14:09:48.045193 13313 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 6d6516b6f77b4389aa53862668486bba from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:48871)
I20240501 14:09:48.045280 13313 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 6d6516b6f77b4389aa53862668486bba from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:48871: session id = 2986bfcc5da34cbca2cad1643b6e35c9-6d6516b6f77b4389aa53862668486bba
I20240501 14:09:48.045995 15776 tablet_copy_client.cc:1002] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.046012 13313 tablet_copy_source_session.cc:205] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.046766 15785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6516b6f77b4389aa53862668486bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.048588 15785 tablet_copy_client.cc:783] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.048714 15785 tablet_copy_client.cc:647] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.049592 15786 tablet_copy_client.cc:1002] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.053133 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.201:0
--local_ip_for_outbound_sockets=127.6.153.201
--webserver_interface=127.6.153.201
--webserver_port=0
--tserver_master_addrs=127.6.153.254:41367
--builtin_ntp_servers=127.6.153.212:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 14:09:48.055428 15797 ts_tablet_manager.cc:902] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:48.055979 15797 tablet_copy_client.cc:308] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:48.056463 13313 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 5583f4e9fe9741649d365daa521355c8 from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:48871)
I20240501 14:09:48.056553 13313 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 5583f4e9fe9741649d365daa521355c8 from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:48871: session id = 2986bfcc5da34cbca2cad1643b6e35c9-5583f4e9fe9741649d365daa521355c8
I20240501 14:09:48.057379 13313 tablet_copy_source_session.cc:205] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.058327 15797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5583f4e9fe9741649d365daa521355c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.060281 15797 tablet_copy_client.cc:783] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.060472 15797 tablet_copy_client.cc:647] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.061736 15799 tablet_copy_client.cc:1002] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.098425 15822 ts_tablet_manager.cc:902] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:48.099133 15822 tablet_copy_client.cc:308] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:48.099694 13313 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 2a19716d4a9b4acd8a3515b48d9eac32 from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:41149)
I20240501 14:09:48.099804 13313 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 2a19716d4a9b4acd8a3515b48d9eac32 from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:41149: session id = 70c63e471e5b482b8b257c3e7543feb5-2a19716d4a9b4acd8a3515b48d9eac32
I20240501 14:09:48.100592 13313 tablet_copy_source_session.cc:205] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.101505 15822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a19716d4a9b4acd8a3515b48d9eac32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.103842 15822 tablet_copy_client.cc:783] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.103991 15822 tablet_copy_client.cc:647] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.104939 15826 tablet_copy_client.cc:1002] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.156018 15877 ts_tablet_manager.cc:902] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:48.156500 15877 tablet_copy_client.cc:308] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:48.157019 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 627fd32202d247c2af434ab8be152102 from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:58259)
I20240501 14:09:48.157120 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 627fd32202d247c2af434ab8be152102 from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:58259: session id = 18bef23ddc624bd784ea38fc6ef00040-627fd32202d247c2af434ab8be152102
I20240501 14:09:48.157760 12917 tablet_copy_source_session.cc:205] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.158586 15877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627fd32202d247c2af434ab8be152102. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.160812 15877 tablet_copy_client.cc:783] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.160952 15877 tablet_copy_client.cc:647] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.162483 15879 tablet_copy_client.cc:1002] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
W20240501 14:09:48.180696 15796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:09:48.180871 15796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:09:48.180927 15796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:09:48.184547 15796 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:09:48.184629 15796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:09:48.184726 15796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.201
I20240501 14:09:48.188361 15796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.201:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.6.153.201
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.6.153.254:41367
--never_fsync=true
--heap_profile_path=/tmp/kudu.15796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.201
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:09:48.189111 15796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:09:48.190147 15796 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240501 14:09:48.193055 15910 ts_tablet_manager.cc:902] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:48.193513 15910 tablet_copy_client.cc:308] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:48.193935 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet 56314112ecb94098950fb77e1dd233aa from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:42337)
I20240501 14:09:48.194021 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet 56314112ecb94098950fb77e1dd233aa from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:42337: session id = 3025ea3ec516466fbe5c5c4337733584-56314112ecb94098950fb77e1dd233aa
I20240501 14:09:48.194711 13049 tablet_copy_source_session.cc:205] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.195550 15910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56314112ecb94098950fb77e1dd233aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.197485 15910 tablet_copy_client.cc:783] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.197638 15910 tablet_copy_client.cc:647] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
W20240501 14:09:48.198132 15917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:09:48.198436 15916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:48.198695 15914 tablet_copy_client.cc:1002] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
W20240501 14:09:48.198825 15919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:09:48.199216 15796 server_base.cc:1034] running on GCE node
I20240501 14:09:48.199563 15796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:09:48.200009 15796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:09:48.201153 15796 hybrid_clock.cc:648] HybridClock initialized: now 1714572588201142 us; error 26 us; skew 500 ppm
I20240501 14:09:48.202973 15796 webserver.cc:460] Webserver started at http://127.6.153.201:42679/ using document root <none> and password file <none>
I20240501 14:09:48.203523 15796 fs_manager.cc:359] Metadata directory not provided
I20240501 14:09:48.203580 15796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:09:48.203737 15796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:09:48.204046 15924 ts_tablet_manager.cc:902] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:48.204476 15924 tablet_copy_client.cc:308] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:48.205367 15796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "777595e1371c421fb350f0bfbb7a8035"
format_stamp: "Formatted at 2024-05-01 14:09:48 on dist-test-slave-cmf0"
I20240501 14:09:48.205793 15796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "777595e1371c421fb350f0bfbb7a8035"
format_stamp: "Formatted at 2024-05-01 14:09:48 on dist-test-slave-cmf0"
I20240501 14:09:48.207574 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet f63509c2039d49f09eee28a45e03ad81 from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:58931)
I20240501 14:09:48.207688 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet f63509c2039d49f09eee28a45e03ad81 from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:58931: session id = 2986bfcc5da34cbca2cad1643b6e35c9-f63509c2039d49f09eee28a45e03ad81
I20240501 14:09:48.208407 13181 tablet_copy_source_session.cc:205] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.209028 15796 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:09:48.209128 15924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63509c2039d49f09eee28a45e03ad81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.210945 15924 tablet_copy_client.cc:783] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.211091 15924 tablet_copy_client.cc:647] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.211292 15930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:48.211985 15928 tablet_copy_client.cc:1002] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.212325 15796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:09:48.212421 15796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "777595e1371c421fb350f0bfbb7a8035"
format_stamp: "Formatted at 2024-05-01 14:09:48 on dist-test-slave-cmf0"
I20240501 14:09:48.212492 15796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:09:48.216230 15931 ts_tablet_manager.cc:902] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:48.216697 15931 tablet_copy_client.cc:308] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:48.220258 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet d8ea7bbf6eec49728d69ec4b6e825e1e from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:36507)
I20240501 14:09:48.220350 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet d8ea7bbf6eec49728d69ec4b6e825e1e from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:36507: session id = 18bef23ddc624bd784ea38fc6ef00040-d8ea7bbf6eec49728d69ec4b6e825e1e
I20240501 14:09:48.221101 13049 tablet_copy_source_session.cc:205] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:48.221825 15931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8ea7bbf6eec49728d69ec4b6e825e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:48.223661 15931 tablet_copy_client.cc:783] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:48.223800 15931 tablet_copy_client.cc:647] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:48.224627 15938 tablet_copy_client.cc:1002] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:48.232354 15796 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:09:48.232826 15796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:09:48.232944 15796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:09:48.233381 15796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:09:48.234189 15796 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:09:48.234233 15796 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:48.234272 15796 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:09:48.234298 15796 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:09:48.242063 15796 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.201:41197
I20240501 14:09:48.242110 16048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.201:41197 every 8 connection(s)
I20240501 14:09:48.242905 15796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 14:09:48.249928 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 15796
I20240501 14:09:48.250056 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240501 14:09:48.250490 16049 heartbeater.cc:347] Connected to a master server at 127.6.153.254:41367
I20240501 14:09:48.250662 16049 heartbeater.cc:464] Registering TS with master...
I20240501 14:09:48.251073 16049 heartbeater.cc:511] Master 127.6.153.254:41367 requested a full tablet report, sending...
I20240501 14:09:48.251945 12747 ts_manager.cc:194] Registered new tserver with Master: 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197)
I20240501 14:09:48.252450 12747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.201:49405
I20240501 14:09:49.254027 16049 heartbeater.cc:503] Master 127.6.153.254:41367 was elected leader, sending a full tablet report...
I20240501 14:09:49.356956 15667 tablet_copy_client.cc:523] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.359045 15667 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:49.365593 15679 tablet_copy_client.cc:523] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.365574 15667 log.cc:824] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:49.367295 15679 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:49.371423 15688 tablet_copy_client.cc:523] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.372629 15688 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:49.372859 15679 log.cc:824] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:49.397193 15688 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.398131 15688 tablet_bootstrap.cc:492] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:49.398432 15688 ts_tablet_manager.cc:1367] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.026s user 0.023s sys 0.000s
I20240501 14:09:49.400713 15679 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.401013 15679 tablet_bootstrap.cc:492] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:49.401268 15679 ts_tablet_manager.cc:1367] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.034s user 0.032s sys 0.000s
I20240501 14:09:49.402639 15667 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.402928 15667 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:49.403128 15667 ts_tablet_manager.cc:1367] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.044s user 0.025s sys 0.005s
I20240501 14:09:49.403513 15679 raft_consensus.cc:348] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.403633 15667 raft_consensus.cc:348] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.403735 15667 raft_consensus.cc:729] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:49.403736 15679 raft_consensus.cc:729] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:49.404068 15679 consensus_queue.cc:260] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.402803 15688 raft_consensus.cc:348] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.404369 15688 raft_consensus.cc:729] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:49.404852 15679 ts_tablet_manager.cc:1398] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 14:09:49.404795 15688 consensus_queue.cc:260] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.406034 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-68927b876c634175841f0859e3f535f4 received from {username='slave'} at 127.6.153.200:42337
I20240501 14:09:49.406132 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-68927b876c634175841f0859e3f535f4 on tablet 68927b876c634175841f0859e3f535f4 with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:49.406883 15688 ts_tablet_manager.cc:1398] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20240501 14:09:49.406942 15700 tablet_copy_client.cc:523] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.404068 15667 consensus_queue.cc:260] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.408497 15667 ts_tablet_manager.cc:1398] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:09:49.408871 15700 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:49.411789 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-37047d45ffe14570b28e459afdb66d7e received from {username='slave'} at 127.6.153.199:41969
I20240501 14:09:49.411882 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-37047d45ffe14570b28e459afdb66d7e on tablet 37047d45ffe14570b28e459afdb66d7e with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:49.412307 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-439e22f89e944051b44d3263c43b2fc3 received from {username='slave'} at 127.6.153.199:52497
I20240501 14:09:49.412401 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-439e22f89e944051b44d3263c43b2fc3 on tablet 439e22f89e944051b44d3263c43b2fc3 with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:49.414665 15700 log.cc:824] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:49.432885 15700 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.433135 15700 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:49.433351 15700 ts_tablet_manager.cc:1367] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.025s user 0.021s sys 0.001s
I20240501 14:09:49.434952 15700 raft_consensus.cc:348] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.435153 15700 raft_consensus.cc:729] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:49.435442 15700 consensus_queue.cc:260] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.436143 15700 ts_tablet_manager.cc:1398] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20240501 14:09:49.436846 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-fe046c56dde144f9adbbbf779067476e received from {username='slave'} at 127.6.153.197:57299
I20240501 14:09:49.436995 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-fe046c56dde144f9adbbbf779067476e on tablet fe046c56dde144f9adbbbf779067476e with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:49.445729 15739 tablet_copy_client.cc:523] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.446830 15739 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:49.449978 15733 tablet_copy_client.cc:523] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.451086 15734 tablet_copy_client.cc:523] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.451392 15727 tablet_copy_client.cc:523] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.451767 15733 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:49.452214 15734 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:49.452548 15727 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:49.457036 15727 log.cc:824] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:49.468415 15748 tablet_copy_client.cc:523] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.469596 15748 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:49.474951 15739 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.475239 15739 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:49.475441 15739 ts_tablet_manager.cc:1367] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20240501 14:09:49.475967 15739 raft_consensus.cc:348] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.476135 15739 raft_consensus.cc:729] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:49.476279 15739 consensus_queue.cc:260] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.476581 15739 ts_tablet_manager.cc:1398] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:49.477062 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-7e34fd488c214188aac9990af9de0db4 received from {username='slave'} at 127.6.153.200:59777
I20240501 14:09:49.477145 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-7e34fd488c214188aac9990af9de0db4 on tablet 7e34fd488c214188aac9990af9de0db4 with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:49.481173 15734 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.481523 15734 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:49.481684 15734 ts_tablet_manager.cc:1367] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.030s user 0.027s sys 0.000s
I20240501 14:09:49.482164 15734 raft_consensus.cc:348] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.482293 15734 raft_consensus.cc:729] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:49.482445 15734 consensus_queue.cc:260] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.482728 15734 ts_tablet_manager.cc:1398] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:49.482966 15727 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.483223 15727 tablet_bootstrap.cc:492] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:49.483457 13313 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-15661c749d7a4cd688d4d666a5eca1c8 received from {username='slave'} at 127.6.153.197:48871
I20240501 14:09:49.483553 13313 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-15661c749d7a4cd688d4d666a5eca1c8 on tablet 15661c749d7a4cd688d4d666a5eca1c8 with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:49.483496 15727 ts_tablet_manager.cc:1367] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.031s user 0.028s sys 0.004s
I20240501 14:09:49.485134 15733 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.485410 15733 tablet_bootstrap.cc:492] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:49.485630 15733 ts_tablet_manager.cc:1367] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.034s user 0.029s sys 0.000s
I20240501 14:09:49.486056 15727 raft_consensus.cc:348] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.486088 15733 raft_consensus.cc:348] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.486258 15727 raft_consensus.cc:729] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:49.486255 15733 raft_consensus.cc:729] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:49.486584 15727 consensus_queue.cc:260] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.486583 15733 consensus_queue.cc:260] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.487370 15733 ts_tablet_manager.cc:1398] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:49.487370 15727 ts_tablet_manager.cc:1398] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20240501 14:09:49.487991 13313 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-78408b0cc50c438caf75876d8dcf88a3 received from {username='slave'} at 127.6.153.198:49463
I20240501 14:09:49.488065 13313 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-78408b0cc50c438caf75876d8dcf88a3 on tablet 78408b0cc50c438caf75876d8dcf88a3 with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:49.488829 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-ecd37c11bf4c45de95f861c1eb4f20d6 received from {username='slave'} at 127.6.153.198:59809
I20240501 14:09:49.488901 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-ecd37c11bf4c45de95f861c1eb4f20d6 on tablet ecd37c11bf4c45de95f861c1eb4f20d6 with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:49.499763 14928 raft_consensus.cc:1203] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.500062 15748 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.500289 15748 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:49.500439 15748 ts_tablet_manager.cc:1367] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.000s
I20240501 14:09:49.500800 15748 raft_consensus.cc:348] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.500901 15748 raft_consensus.cc:729] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:49.500991 15748 consensus_queue.cc:260] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.501303 15748 ts_tablet_manager.cc:1398] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:49.501768 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-c2affe37c7174ff1be13ba4807d1fffe received from {username='slave'} at 127.6.153.200:59777
I20240501 14:09:49.501832 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-c2affe37c7174ff1be13ba4807d1fffe on tablet c2affe37c7174ff1be13ba4807d1fffe with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:49.510831 14797 raft_consensus.cc:1203] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.511538 15060 raft_consensus.cc:1203] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.527643 15764 tablet_copy_client.cc:523] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.528695 15764 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:49.533303 15769 tablet_copy_client.cc:523] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.534649 15769 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:49.543882 15779 tablet_copy_client.cc:523] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.545423 15779 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:49.546244 15774 tablet_copy_client.cc:523] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.547509 15774 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:49.549888 15785 tablet_copy_client.cc:523] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.551098 15785 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:49.552348 15764 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.552664 15764 tablet_bootstrap.cc:492] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:49.552870 15764 ts_tablet_manager.cc:1367] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.024s user 0.019s sys 0.005s
I20240501 14:09:49.553292 15764 raft_consensus.cc:348] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.553458 15764 raft_consensus.cc:729] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:49.553591 15764 consensus_queue.cc:260] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.553879 15764 ts_tablet_manager.cc:1398] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:49.554457 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-6e2d06a2ae3f4fe5a515d142d3e608e8 received from {username='slave'} at 127.6.153.200:55411
I20240501 14:09:49.554549 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-6e2d06a2ae3f4fe5a515d142d3e608e8 on tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:49.562016 15797 tablet_copy_client.cc:523] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.563283 15797 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:49.564049 15769 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.564321 15769 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:49.564468 15769 ts_tablet_manager.cc:1367] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.030s user 0.019s sys 0.004s
I20240501 14:09:49.564872 15769 raft_consensus.cc:348] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.564991 15769 raft_consensus.cc:729] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:49.565083 15769 consensus_queue.cc:260] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.566992 15769 ts_tablet_manager.cc:1398] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:09:49.568691 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-4fc05e3adc8e4ed0b45d275725f289f4 received from {username='slave'} at 127.6.153.198:58259
I20240501 14:09:49.568759 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-4fc05e3adc8e4ed0b45d275725f289f4 on tablet 4fc05e3adc8e4ed0b45d275725f289f4 with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:49.574543 14665 raft_consensus.cc:1203] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.577463 15779 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.577769 15779 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:49.577991 15779 ts_tablet_manager.cc:1367] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.033s user 0.026s sys 0.000s
I20240501 14:09:49.578418 15779 raft_consensus.cc:348] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.578579 15779 raft_consensus.cc:729] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:49.578697 15779 consensus_queue.cc:260] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.579035 15779 ts_tablet_manager.cc:1398] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:49.579521 13313 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-34f677061e42436ab08123227f2be7bf received from {username='slave'} at 127.6.153.199:41149
I20240501 14:09:49.579597 13313 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-34f677061e42436ab08123227f2be7bf on tablet 34f677061e42436ab08123227f2be7bf with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:49.580358 15774 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.580721 15774 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:49.580904 15774 ts_tablet_manager.cc:1367] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.033s user 0.024s sys 0.001s
I20240501 14:09:49.581321 15774 raft_consensus.cc:348] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.581637 15774 raft_consensus.cc:729] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:49.581794 15774 consensus_queue.cc:260] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.582162 15774 ts_tablet_manager.cc:1398] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:49.582662 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-5053c0f554af4efaa221444af87dd9b9 received from {username='slave'} at 127.6.153.199:52399
I20240501 14:09:49.582737 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-5053c0f554af4efaa221444af87dd9b9 on tablet 5053c0f554af4efaa221444af87dd9b9 with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:49.591037 15797 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.591259 15797 tablet_bootstrap.cc:492] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:49.591390 15785 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.591421 15797 ts_tablet_manager.cc:1367] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.028s user 0.018s sys 0.005s
I20240501 14:09:49.591606 15785 tablet_bootstrap.cc:492] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:49.591730 15785 ts_tablet_manager.cc:1367] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.041s user 0.022s sys 0.000s
I20240501 14:09:49.591861 15797 raft_consensus.cc:348] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.591972 15797 raft_consensus.cc:729] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:49.592051 15785 raft_consensus.cc:348] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.592165 15785 raft_consensus.cc:729] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:49.592295 15785 consensus_queue.cc:260] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.592123 15797 consensus_queue.cc:260] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.592588 15785 ts_tablet_manager.cc:1398] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:49.592661 15797 ts_tablet_manager.cc:1398] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:49.593019 13313 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-6d6516b6f77b4389aa53862668486bba received from {username='slave'} at 127.6.153.197:48871
I20240501 14:09:49.593099 13313 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-6d6516b6f77b4389aa53862668486bba on tablet 6d6516b6f77b4389aa53862668486bba with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:49.593251 13312 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-5583f4e9fe9741649d365daa521355c8 received from {username='slave'} at 127.6.153.197:48871
I20240501 14:09:49.593318 13312 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-5583f4e9fe9741649d365daa521355c8 on tablet 5583f4e9fe9741649d365daa521355c8 with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:49.605165 15822 tablet_copy_client.cc:523] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.606279 15822 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:49.623795 15822 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.624047 15822 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:49.624217 15822 ts_tablet_manager.cc:1367] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.018s user 0.018s sys 0.000s
I20240501 14:09:49.624609 15822 raft_consensus.cc:348] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.624728 15822 raft_consensus.cc:729] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:49.624827 15822 consensus_queue.cc:260] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:49.625069 15822 ts_tablet_manager.cc:1398] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:49.625545 13312 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-2a19716d4a9b4acd8a3515b48d9eac32 received from {username='slave'} at 127.6.153.199:41149
I20240501 14:09:49.625618 13312 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-2a19716d4a9b4acd8a3515b48d9eac32 on tablet 2a19716d4a9b4acd8a3515b48d9eac32 with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:49.662760 15877 tablet_copy_client.cc:523] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.664047 15877 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:49.685671 15877 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.686024 15877 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:49.686180 15877 ts_tablet_manager.cc:1367] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.022s user 0.016s sys 0.006s
I20240501 14:09:49.686554 15877 raft_consensus.cc:348] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.686666 15877 raft_consensus.cc:729] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:49.686754 15877 consensus_queue.cc:260] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.686985 15877 ts_tablet_manager.cc:1398] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:49.687498 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-627fd32202d247c2af434ab8be152102 received from {username='slave'} at 127.6.153.198:58259
I20240501 14:09:49.687578 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-627fd32202d247c2af434ab8be152102 on tablet 627fd32202d247c2af434ab8be152102 with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:49.699012 15910 tablet_copy_client.cc:523] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.700244 15910 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:49.712242 15924 tablet_copy_client.cc:523] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.713321 15924 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:49.724872 15931 tablet_copy_client.cc:523] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:49.725419 15910 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.725836 15910 tablet_bootstrap.cc:492] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:49.725999 15910 ts_tablet_manager.cc:1367] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20240501 14:09:49.726212 15931 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:49.726420 15910 raft_consensus.cc:348] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.726537 15910 raft_consensus.cc:729] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:49.726634 15910 consensus_queue.cc:260] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:49.726887 15910 ts_tablet_manager.cc:1398] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:49.727382 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-56314112ecb94098950fb77e1dd233aa received from {username='slave'} at 127.6.153.200:42337
I20240501 14:09:49.727454 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-56314112ecb94098950fb77e1dd233aa on tablet 56314112ecb94098950fb77e1dd233aa with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:49.734786 15924 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.734995 15924 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:49.735136 15924 ts_tablet_manager.cc:1367] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.022s user 0.018s sys 0.000s
I20240501 14:09:49.735533 15924 raft_consensus.cc:348] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.735630 15924 raft_consensus.cc:729] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:49.735709 15924 consensus_queue.cc:260] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:49.736028 15924 ts_tablet_manager.cc:1398] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240501 14:09:49.736513 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-f63509c2039d49f09eee28a45e03ad81 received from {username='slave'} at 127.6.153.197:58931
I20240501 14:09:49.736594 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-f63509c2039d49f09eee28a45e03ad81 on tablet f63509c2039d49f09eee28a45e03ad81 with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:49.748296 15931 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:49.748549 15931 tablet_bootstrap.cc:492] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:49.748709 15931 ts_tablet_manager.cc:1367] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.023s user 0.016s sys 0.004s
I20240501 14:09:49.749135 15931 raft_consensus.cc:348] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.749253 15931 raft_consensus.cc:729] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:49.749349 15931 consensus_queue.cc:260] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:49.749631 15931 ts_tablet_manager.cc:1398] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20240501 14:09:49.750332 14797 raft_consensus.cc:1203] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.750510 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-d8ea7bbf6eec49728d69ec4b6e825e1e received from {username='slave'} at 127.6.153.198:36507
I20240501 14:09:49.750587 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-d8ea7bbf6eec49728d69ec4b6e825e1e on tablet d8ea7bbf6eec49728d69ec4b6e825e1e with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:49.754719 14665 raft_consensus.cc:1203] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.780920 14928 raft_consensus.cc:1203] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.815246 14928 raft_consensus.cc:1203] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.847254 15060 raft_consensus.cc:1203] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.859863 14797 raft_consensus.cc:1203] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.883148 14665 raft_consensus.cc:1203] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.889648 15060 raft_consensus.cc:1203] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.904340 14665 raft_consensus.cc:1203] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.905267 14665 raft_consensus.cc:1203] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.932806 14797 raft_consensus.cc:1203] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:49.971105 15060 raft_consensus.cc:1203] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:50.000028 16641 raft_consensus.cc:1050] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:50.000499 16641 consensus_queue.cc:237] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.001868 13028 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.002149 13161 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.002276 16071 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.003119 14665 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.003545 16641 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.003873 16071 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.006253 16071 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.008258 13028 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.008797 12745 catalog_manager.cc:5484] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.009896 14665 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.010334 13161 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.013064 14928 raft_consensus.cc:1203] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:50.019955 13292 consensus_queue.cc:237] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.021056 14665 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.021057 13028 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.021453 16641 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.021620 16477 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.022979 16477 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.024421 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 15661c749d7a4cd688d4d666a5eca1c8 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.025061 14665 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.025568 12748 catalog_manager.cc:5484] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.025578 13028 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.028465 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 15661c749d7a4cd688d4d666a5eca1c8 with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:50.028924 16666 tablet_replica.cc:330] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:50.029326 16666 raft_consensus.cc:2229] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.029696 16666 raft_consensus.cc:2258] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.030339 15060 raft_consensus.cc:1203] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:50.031531 16666 ts_tablet_manager.cc:1875] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.033699 16666 ts_tablet_manager.cc:1888] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.033789 16666 log.cc:1196] T 15661c749d7a4cd688d4d666a5eca1c8 P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/15661c749d7a4cd688d4d666a5eca1c8
I20240501 14:09:50.034348 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 15661c749d7a4cd688d4d666a5eca1c8 (table rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]) successfully deleted
I20240501 14:09:50.054677 16693 raft_consensus.cc:1050] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:50.055081 16693 consensus_queue.cc:237] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.056305 13161 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.056479 14928 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.056957 13292 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.057063 16692 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.057204 16693 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.057938 16692 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.060786 16692 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.061576 14928 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.062570 13293 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.064165 12747 catalog_manager.cc:5484] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.068789 13161 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.072475 12897 consensus_queue.cc:237] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.073477 13293 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.073647 14928 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.073949 16693 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.074102 16692 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.075062 12897 consensus_queue.cc:237] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:50.076210 15214 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.076324 14928 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.076439 13028 raft_consensus.cc:1261] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.077481 15214 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.078027 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 37047d45ffe14570b28e459afdb66d7e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.078301 12747 catalog_manager.cc:5484] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.078418 13161 raft_consensus.cc:1261] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.078862 16692 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.079042 13293 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
W20240501 14:09:50.080469 12831 consensus_peers.cc:485] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a948907bfb14d0d958ebb969e5a6a50. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.080436 15214 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.080703 13028 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.081941 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 37047d45ffe14570b28e459afdb66d7e with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:50.082444 16666 tablet_replica.cc:330] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:50.082579 16666 raft_consensus.cc:2229] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.082715 16666 raft_consensus.cc:2258] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.083408 12745 catalog_manager.cc:5484] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.083873 16666 ts_tablet_manager.cc:1875] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.085631 16666 ts_tablet_manager.cc:1888] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.085691 16666 log.cc:1196] T 37047d45ffe14570b28e459afdb66d7e P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/37047d45ffe14570b28e459afdb66d7e
I20240501 14:09:50.086091 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 37047d45ffe14570b28e459afdb66d7e (table rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]) successfully deleted
I20240501 14:09:50.088179 13161 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.088541 16665 raft_consensus.cc:1050] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:50.088871 16665 consensus_queue.cc:237] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.090029 13028 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.090350 14797 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.090373 16071 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.090682 13159 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.091190 16477 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.093037 16071 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.093966 16477 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.094671 14797 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.095211 13159 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.096225 13028 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.096853 12748 catalog_manager.cc:5484] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.100690 13293 consensus_queue.cc:237] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.101724 13159 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.101898 14797 raft_consensus.cc:1261] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.102097 16641 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.102250 16071 consensus_queue.cc:1035] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.103803 16665 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.104163 13159 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.104583 14797 raft_consensus.cc:2932] T 78408b0cc50c438caf75876d8dcf88a3 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.105513 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 78408b0cc50c438caf75876d8dcf88a3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.107005 12745 catalog_manager.cc:5484] T 78408b0cc50c438caf75876d8dcf88a3 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.109632 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 78408b0cc50c438caf75876d8dcf88a3 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:50.110502 16734 tablet_replica.cc:330] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:50.110757 16734 raft_consensus.cc:2229] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.110935 16734 raft_consensus.cc:2258] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.111939 16734 ts_tablet_manager.cc:1875] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.113322 16735 raft_consensus.cc:1050] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:50.113718 16735 consensus_queue.cc:237] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.115134 16734 ts_tablet_manager.cc:1888] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.115260 16734 log.cc:1196] T 78408b0cc50c438caf75876d8dcf88a3 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/78408b0cc50c438caf75876d8dcf88a3
I20240501 14:09:50.115281 15060 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.115700 13293 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.115763 15418 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.115978 12897 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.116271 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 78408b0cc50c438caf75876d8dcf88a3 (table rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]) successfully deleted
I20240501 14:09:50.116597 16735 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.116883 16735 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.118749 15418 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.119242 12897 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.120898 13293 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.122040 12752 catalog_manager.cc:5484] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.122603 15060 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.126039 13028 consensus_queue.cc:237] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.126955 15060 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.127033 13293 raft_consensus.cc:1261] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.127367 16735 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.127513 16742 consensus_queue.cc:1035] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.128903 15418 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.129377 15060 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.129628 13293 raft_consensus.cc:2932] T 68927b876c634175841f0859e3f535f4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.131258 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 68927b876c634175841f0859e3f535f4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.131677 12752 catalog_manager.cc:5484] T 68927b876c634175841f0859e3f535f4 P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.133504 16742 raft_consensus.cc:1050] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:50.133842 16742 consensus_queue.cc:237] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.135140 12897 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.135172 14797 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.135172 13293 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.135141 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 68927b876c634175841f0859e3f535f4 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:50.135550 15418 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.135701 16735 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.135895 16742 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.135939 16753 tablet_replica.cc:330] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:50.136240 16753 raft_consensus.cc:2229] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.136462 16753 raft_consensus.cc:2258] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.138032 16753 ts_tablet_manager.cc:1875] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.138417 16735 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.138792 14797 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.140321 16753 ts_tablet_manager.cc:1888] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.140414 16753 log.cc:1196] T 68927b876c634175841f0859e3f535f4 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/68927b876c634175841f0859e3f535f4
I20240501 14:09:50.140713 12747 catalog_manager.cc:5484] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.140976 12897 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.141362 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 68927b876c634175841f0859e3f535f4 (table rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]) successfully deleted
I20240501 14:09:50.142971 13293 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.143838 13028 consensus_queue.cc:237] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.145159 12897 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.145242 14797 raft_consensus.cc:1261] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.145609 15418 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.145771 16742 consensus_queue.cc:1035] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.147308 16735 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.147907 12897 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.147913 14797 raft_consensus.cc:2932] T d8ea7bbf6eec49728d69ec4b6e825e1e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.148666 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d8ea7bbf6eec49728d69ec4b6e825e1e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.149917 12745 catalog_manager.cc:5484] T d8ea7bbf6eec49728d69ec4b6e825e1e P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.151520 14797 raft_consensus.cc:1203] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:50.152247 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet d8ea7bbf6eec49728d69ec4b6e825e1e with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:50.153055 16767 tablet_replica.cc:330] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:50.153553 16767 raft_consensus.cc:2229] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.153892 16767 raft_consensus.cc:2258] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.155588 16767 ts_tablet_manager.cc:1875] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.157557 16767 ts_tablet_manager.cc:1888] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.157630 16767 log.cc:1196] T d8ea7bbf6eec49728d69ec4b6e825e1e P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/d8ea7bbf6eec49728d69ec4b6e825e1e
I20240501 14:09:50.158175 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet d8ea7bbf6eec49728d69ec4b6e825e1e (table rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]) successfully deleted
I20240501 14:09:50.167918 14928 raft_consensus.cc:1203] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:50.227413 13028 consensus_queue.cc:237] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:50.228684 13159 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.228686 12897 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.229076 15418 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.229243 16742 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 14:09:50.229437 12962 consensus_peers.cc:485] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73407db0172f454baf2bb44a85add182. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.230839 16742 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:50.231245 12897 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:50.231691 13159 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:50.233361 12747 catalog_manager.cc:5484] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.240634 12897 consensus_queue.cc:237] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:50.241886 13028 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.242004 13293 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:09:50.242184 12833 consensus_peers.cc:485] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ca90427253c4320b2fea78e3fc89d92. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.242348 16692 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.242493 16693 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.244343 16710 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:50.245635 13028 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:50.245719 13292 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:50.248001 12745 catalog_manager.cc:5484] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:50.253674 13159 consensus_queue.cc:237] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:50.254760 13292 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.254870 12897 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:09:50.254961 13096 consensus_peers.cc:485] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ac389d29e274b15b5f3db598a04f5e7. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.255178 16856 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.255358 14343 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.256837 16052 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:50.257355 13292 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:50.258534 12896 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:50.260582 12752 catalog_manager.cc:5484] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:50.315742 15418 raft_consensus.cc:1050] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:50.316125 15418 consensus_queue.cc:237] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.317538 12896 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.317653 14928 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.317642 13292 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.317874 16735 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.318017 16742 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.318231 16752 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.320436 16735 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.321928 12896 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.323948 12747 catalog_manager.cc:5484] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.327802 13292 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.327931 14928 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.330824 13028 consensus_queue.cc:237] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.332088 12896 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.332521 16752 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.333207 14928 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.333496 16752 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.334057 15418 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.334458 12896 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.335422 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5053c0f554af4efaa221444af87dd9b9 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.336815 12752 catalog_manager.cc:5484] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.338044 16856 raft_consensus.cc:1050] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:50.338508 16856 consensus_queue.cc:237] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.338892 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 5053c0f554af4efaa221444af87dd9b9 with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:50.339069 16767 tablet_replica.cc:330] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:50.339200 16767 raft_consensus.cc:2229] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.339365 16767 raft_consensus.cc:2258] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.339797 14928 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.339989 14665 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.340530 16767 ts_tablet_manager.cc:1875] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.341078 16859 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.342504 12896 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.342644 16767 ts_tablet_manager.cc:1888] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.342690 13028 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.342708 16767 log.cc:1196] T 5053c0f554af4efaa221444af87dd9b9 P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/5053c0f554af4efaa221444af87dd9b9
I20240501 14:09:50.343053 16859 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.343206 14343 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.343629 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 5053c0f554af4efaa221444af87dd9b9 (table rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]) successfully deleted
I20240501 14:09:50.345255 16859 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.346948 12896 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.347281 16477 raft_consensus.cc:1050] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:50.347663 16477 consensus_queue.cc:237] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.348927 12896 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.349232 14665 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.349282 16665 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.349337 12745 catalog_manager.cc:5484] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.349469 13028 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.349668 14664 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.350809 13029 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.351305 16641 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.353161 16641 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.354815 13159 consensus_queue.cc:237] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.355109 16641 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.355700 12896 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.355860 13028 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.356225 16052 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.356266 14664 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.356652 14343 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.358208 13028 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.358304 12752 catalog_manager.cc:5484] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.358873 16052 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.359256 14664 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.360236 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f63509c2039d49f09eee28a45e03ad81 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.361277 12748 catalog_manager.cc:5484] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.361495 14665 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.363449 13292 consensus_queue.cc:237] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.364676 13028 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.364858 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet f63509c2039d49f09eee28a45e03ad81 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:50.365028 16753 tablet_replica.cc:330] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:50.365093 14664 raft_consensus.cc:1261] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.365144 16753 raft_consensus.cc:2229] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.365279 16753 raft_consensus.cc:2258] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.365387 16477 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.365676 16071 consensus_queue.cc:1035] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.366037 16753 ts_tablet_manager.cc:1875] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.365947 13028 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.367674 16071 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.367971 16753 ts_tablet_manager.cc:1888] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.368027 16753 log.cc:1196] T f63509c2039d49f09eee28a45e03ad81 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/f63509c2039d49f09eee28a45e03ad81
I20240501 14:09:50.368476 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet f63509c2039d49f09eee28a45e03ad81 (table rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]) successfully deleted
I20240501 14:09:50.368969 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5583f4e9fe9741649d365daa521355c8 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.369304 14665 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.370188 13029 raft_consensus.cc:2932] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.372509 12748 catalog_manager.cc:5484] T 5583f4e9fe9741649d365daa521355c8 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.374660 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 5583f4e9fe9741649d365daa521355c8 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:50.374943 16753 tablet_replica.cc:330] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:50.375070 16753 raft_consensus.cc:2229] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.375201 16753 raft_consensus.cc:2258] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.376045 16753 ts_tablet_manager.cc:1875] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.377380 16753 ts_tablet_manager.cc:1888] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.377441 16753 log.cc:1196] T 5583f4e9fe9741649d365daa521355c8 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/5583f4e9fe9741649d365daa521355c8
I20240501 14:09:50.377821 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 5583f4e9fe9741649d365daa521355c8 (table rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]) successfully deleted
I20240501 14:09:50.392527 16665 raft_consensus.cc:1050] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:50.392871 16665 consensus_queue.cc:237] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.394155 13159 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.394240 12896 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.394515 16641 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.394698 16477 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.394740 14665 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.395043 16477 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.396723 16742 raft_consensus.cc:1050] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:50.397078 16742 consensus_queue.cc:237] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.398403 13161 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.398561 15060 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.398639 13292 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.398947 16752 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.399082 16735 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.399351 15418 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.400831 16641 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.401275 14665 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.401706 13159 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.402938 12896 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.403705 12748 catalog_manager.cc:5484] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.405453 15418 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.407526 13292 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.407701 12748 catalog_manager.cc:5484] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.408219 13290 consensus_queue.cc:237] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.410229 14665 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.410275 13159 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.410653 16477 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.410795 12896 raft_consensus.cc:1261] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.411623 16477 consensus_queue.cc:1035] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.412811 16477 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.413439 12896 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.413707 15060 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.414232 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6d6516b6f77b4389aa53862668486bba with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.414692 14665 raft_consensus.cc:2932] T 6d6516b6f77b4389aa53862668486bba P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.414856 12747 catalog_manager.cc:5484] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.417035 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d6516b6f77b4389aa53862668486bba with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:50.417337 16666 tablet_replica.cc:330] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:50.417466 16666 raft_consensus.cc:2229] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.417610 16666 raft_consensus.cc:2258] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.418069 13028 consensus_queue.cc:237] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.419275 13290 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.419317 16666 ts_tablet_manager.cc:1875] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.419852 15060 raft_consensus.cc:1261] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.420397 16752 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.420562 16735 consensus_queue.cc:1035] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.421342 16666 ts_tablet_manager.cc:1888] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.421401 16666 log.cc:1196] T 6d6516b6f77b4389aa53862668486bba P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/6d6516b6f77b4389aa53862668486bba
I20240501 14:09:50.422050 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 6d6516b6f77b4389aa53862668486bba (table rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]) successfully deleted
I20240501 14:09:50.422443 16752 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.422909 15060 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.423044 13290 raft_consensus.cc:2932] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.424027 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 56314112ecb94098950fb77e1dd233aa with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.425055 13028 consensus_queue.cc:237] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:50.425282 12745 catalog_manager.cc:5484] T 56314112ecb94098950fb77e1dd233aa P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
W20240501 14:09:50.426478 12962 consensus_peers.cc:485] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 920be22f0370418885ce155d9874607a. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.426558 13159 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.426941 16735 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.427600 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 56314112ecb94098950fb77e1dd233aa with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:50.427925 13290 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.428275 16735 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.428579 16666 tablet_replica.cc:330] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:50.428730 15418 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:50.428857 16666 raft_consensus.cc:2229] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.428985 16666 raft_consensus.cc:2258] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.429546 16666 ts_tablet_manager.cc:1875] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.430557 13159 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:50.431262 16666 ts_tablet_manager.cc:1888] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.431305 16666 log.cc:1196] T 56314112ecb94098950fb77e1dd233aa P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/56314112ecb94098950fb77e1dd233aa
I20240501 14:09:50.431769 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 56314112ecb94098950fb77e1dd233aa (table rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]) successfully deleted
I20240501 14:09:50.431908 12748 catalog_manager.cc:5484] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.433876 16052 raft_consensus.cc:1050] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:50.434120 13290 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:50.434219 16052 consensus_queue.cc:237] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.435437 13290 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.435743 16859 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.436091 13028 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.436888 16859 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.437070 14928 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.437393 16859 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.439612 16052 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.440038 13290 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.440141 14928 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.441802 13028 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.442209 12896 consensus_queue.cc:237] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:50.442339 12747 catalog_manager.cc:5484] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.444023 13159 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.444136 13290 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.444377 16693 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 14:09:50.444630 12833 consensus_peers.cc:485] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f58cd9e96244a2da7954bdaf9272b82. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.445333 16693 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.446260 16692 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:50.446682 13159 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:50.447157 13290 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:50.449259 16711 raft_consensus.cc:1050] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:50.449327 12747 catalog_manager.cc:5484] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index -1 to 4, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.449609 16711 consensus_queue.cc:237] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.450835 13290 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.451107 13028 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.451503 14665 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.451577 16692 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.451745 16693 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.451860 16710 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.454754 16693 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.455114 13028 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.456244 13290 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.456792 14665 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.457253 12747 catalog_manager.cc:5484] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.459854 12896 consensus_queue.cc:237] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:50.460855 14665 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.461153 13028 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.461169 16856 raft_consensus.cc:1050] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:50.461512 16856 consensus_queue.cc:237] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.461687 16692 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.462354 13159 consensus_queue.cc:237] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:50.462853 14797 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.463609 16052 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.463280 12896 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.464182 13029 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.464186 12895 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.464188 16711 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.464740 16859 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.464795 13292 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.465032 16859 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 14:09:50.465315 13095 consensus_peers.cc:485] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 443ae0978f2145658d910e2ad9fa38d0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.465454 16052 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.465881 17012 raft_consensus.cc:1050] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:50.465293 16859 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.466215 17012 consensus_queue.cc:237] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.467542 15060 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.467635 15214 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.467882 16856 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.468098 13290 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.468264 13028 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.468422 16856 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.469432 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fe046c56dde144f9adbbbf779067476e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.469684 12897 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.469945 12745 catalog_manager.cc:5484] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.470140 17012 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.470935 14665 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:50.472054 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet fe046c56dde144f9adbbbf779067476e with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:50.472234 16767 tablet_replica.cc:330] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:50.472352 16767 raft_consensus.cc:2229] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.472466 16767 raft_consensus.cc:2258] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.472406 16859 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.473213 17012 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.474679 12896 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.474835 16052 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.475507 12895 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.476521 14797 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.476665 15060 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.476919 12752 catalog_manager.cc:5484] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.477290 17023 ts_tablet_manager.cc:902] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:50.477746 16767 ts_tablet_manager.cc:1875] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.477747 17023 tablet_copy_client.cc:308] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:50.478296 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 8a948907bfb14d0d958ebb969e5a6a50 from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:57299)
I20240501 14:09:50.478379 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 8a948907bfb14d0d958ebb969e5a6a50 from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:57299: session id = 2986bfcc5da34cbca2cad1643b6e35c9-8a948907bfb14d0d958ebb969e5a6a50
I20240501 14:09:50.478644 12896 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.479210 12917 tablet_copy_source_session.cc:205] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:50.479658 16767 ts_tablet_manager.cc:1888] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.479725 16767 log.cc:1196] T fe046c56dde144f9adbbbf779067476e P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/fe046c56dde144f9adbbbf779067476e
I20240501 14:09:50.480286 17023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a948907bfb14d0d958ebb969e5a6a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:50.480351 12747 catalog_manager.cc:5484] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.481830 13292 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.482208 17023 tablet_copy_client.cc:783] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:50.482332 17023 tablet_copy_client.cc:647] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:50.482352 12745 catalog_manager.cc:5484] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.482602 13029 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.483085 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet fe046c56dde144f9adbbbf779067476e (table rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]) successfully deleted
I20240501 14:09:50.484741 16477 raft_consensus.cc:1050] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:50.485067 16477 consensus_queue.cc:237] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.486505 13029 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.486505 12896 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.486415 13290 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.487036 14928 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.487196 16071 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.487363 16665 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.487566 16641 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.489459 17024 tablet_copy_client.cc:1002] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:50.489697 16665 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.490159 14928 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.491480 13029 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.492023 12896 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.492240 12752 catalog_manager.cc:5484] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.492434 13161 consensus_queue.cc:237] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.493232 13159 consensus_queue.cc:237] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.494823 14797 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.494918 13290 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.495047 13029 raft_consensus.cc:1261] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.495112 15060 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.495231 16856 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.495378 16859 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.495883 16856 consensus_queue.cc:1035] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.496075 16859 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.498689 16700 raft_consensus.cc:1050] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:50.499008 16700 consensus_queue.cc:237] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.499794 16856 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.500067 13290 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.500877 13028 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.501048 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7e34fd488c214188aac9990af9de0db4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.501763 17022 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.502174 12750 catalog_manager.cc:5484] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.502285 14796 raft_consensus.cc:2932] T ecd37c11bf4c45de95f861c1eb4f20d6 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.503041 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ecd37c11bf4c45de95f861c1eb4f20d6 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.503749 12752 catalog_manager.cc:5484] T ecd37c11bf4c45de95f861c1eb4f20d6 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.504722 13028 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.504882 15060 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.505071 16692 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.505896 13159 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.506577 16711 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.509240 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet ecd37c11bf4c45de95f861c1eb4f20d6 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:50.509418 16753 tablet_replica.cc:330] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:50.509536 16753 raft_consensus.cc:2229] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.509672 16753 raft_consensus.cc:2258] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.510241 16753 ts_tablet_manager.cc:1875] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.512048 16753 ts_tablet_manager.cc:1888] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.512115 16753 log.cc:1196] T ecd37c11bf4c45de95f861c1eb4f20d6 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/ecd37c11bf4c45de95f861c1eb4f20d6
I20240501 14:09:50.512388 14796 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.513252 16700 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.513634 15214 raft_consensus.cc:1050] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:50.513969 15214 consensus_queue.cc:237] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.515187 13159 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.515241 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet ecd37c11bf4c45de95f861c1eb4f20d6 (table rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]) successfully deleted
I20240501 14:09:50.515607 16693 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.515728 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e34fd488c214188aac9990af9de0db4 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:50.515885 16753 tablet_replica.cc:330] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:50.516168 15060 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.516381 13028 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.516824 16692 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.517207 16753 raft_consensus.cc:2229] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.517390 16753 raft_consensus.cc:2258] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.517572 16692 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.518085 16753 ts_tablet_manager.cc:1875] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.518321 14796 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.519368 16693 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.520130 13029 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.520290 12748 catalog_manager.cc:5484] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.521136 13161 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.522812 12896 consensus_queue.cc:237] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.524072 13161 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.524094 14796 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.524202 16753 ts_tablet_manager.cc:1888] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.524271 16753 log.cc:1196] T 7e34fd488c214188aac9990af9de0db4 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/7e34fd488c214188aac9990af9de0db4
I20240501 14:09:50.524375 16710 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.524546 16693 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.524694 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 7e34fd488c214188aac9990af9de0db4 (table rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]) successfully deleted
I20240501 14:09:50.525476 16693 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.526139 16700 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.527143 13028 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.527468 15060 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.527949 12745 catalog_manager.cc:5484] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.528470 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4fc05e3adc8e4ed0b45d275725f289f4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.528735 14796 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.529062 16856 raft_consensus.cc:1050] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:50.529136 13160 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.529363 16856 consensus_queue.cc:237] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.530858 12747 catalog_manager.cc:5484] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.531013 13028 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.531368 17022 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.531426 15060 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.531963 17022 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.532007 12896 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.532835 17022 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.533620 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fc05e3adc8e4ed0b45d275725f289f4 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:50.533787 16734 tablet_replica.cc:330] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:50.533901 16734 raft_consensus.cc:2229] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.534032 16734 raft_consensus.cc:2258] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.535002 13160 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.535506 16856 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.536993 16734 ts_tablet_manager.cc:1875] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.537130 13028 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.537732 12896 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.538802 16734 ts_tablet_manager.cc:1888] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.538859 16734 log.cc:1196] T 4fc05e3adc8e4ed0b45d275725f289f4 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/4fc05e3adc8e4ed0b45d275725f289f4
I20240501 14:09:50.539357 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 4fc05e3adc8e4ed0b45d275725f289f4 (table rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]) successfully deleted
I20240501 14:09:50.540253 12752 catalog_manager.cc:5484] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.542886 12896 consensus_queue.cc:237] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.545663 13160 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.545801 15060 raft_consensus.cc:1261] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.545744 15059 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.546172 16692 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.546341 16700 consensus_queue.cc:1035] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.547717 13161 consensus_queue.cc:237] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:50.548058 16710 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.548425 15060 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.550359 12748 catalog_manager.cc:5484] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.551586 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.551666 12896 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.552214 16052 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.552549 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:50.552719 16734 tablet_replica.cc:330] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:50.552832 16734 raft_consensus.cc:2229] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.552956 16734 raft_consensus.cc:2258] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.553557 13161 raft_consensus.cc:2932] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.554018 16734 ts_tablet_manager.cc:1875] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.555907 16734 ts_tablet_manager.cc:1888] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.555984 16734 log.cc:1196] T 6e2d06a2ae3f4fe5a515d142d3e608e8 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/6e2d06a2ae3f4fe5a515d142d3e608e8
I20240501 14:09:50.556413 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 6e2d06a2ae3f4fe5a515d142d3e608e8 (table rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]) successfully deleted
I20240501 14:09:50.556636 15060 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.556973 16052 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.558264 17012 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.558620 15060 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.560132 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c2affe37c7174ff1be13ba4807d1fffe with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.560974 12896 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:50.561255 12750 catalog_manager.cc:5484] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.564563 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet c2affe37c7174ff1be13ba4807d1fffe with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:50.564774 16734 tablet_replica.cc:330] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:50.564879 16734 raft_consensus.cc:2229] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.565006 16734 raft_consensus.cc:2258] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.565606 16734 ts_tablet_manager.cc:1875] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.567611 16734 ts_tablet_manager.cc:1888] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.567724 16734 log.cc:1196] T c2affe37c7174ff1be13ba4807d1fffe P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/c2affe37c7174ff1be13ba4807d1fffe
I20240501 14:09:50.568190 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet c2affe37c7174ff1be13ba4807d1fffe (table rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]) successfully deleted
I20240501 14:09:50.575552 16710 raft_consensus.cc:1050] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:50.575851 16710 consensus_queue.cc:237] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.577039 13161 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.577100 13290 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.577358 14796 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.577380 16711 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.577512 15214 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.578051 16711 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.579959 15214 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.580505 13161 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.581472 13290 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.582295 14796 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.584353 12752 catalog_manager.cc:5484] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.587606 12896 consensus_queue.cc:237] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:50.588552 14796 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.588723 13161 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.588958 16711 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.589102 16710 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.590497 16710 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.591442 13161 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.592260 14796 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:50.592593 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 627fd32202d247c2af434ab8be152102 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.592515 12748 catalog_manager.cc:5484] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.594792 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 627fd32202d247c2af434ab8be152102 with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:50.594970 16767 tablet_replica.cc:330] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:50.595067 16767 raft_consensus.cc:2229] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.595206 16767 raft_consensus.cc:2258] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.595779 16767 ts_tablet_manager.cc:1875] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.597214 16767 ts_tablet_manager.cc:1888] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.597261 16767 log.cc:1196] T 627fd32202d247c2af434ab8be152102 P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/627fd32202d247c2af434ab8be152102
I20240501 14:09:50.597611 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 627fd32202d247c2af434ab8be152102 (table rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]) successfully deleted
I20240501 14:09:50.612541 13161 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3"
dest_uuid: "b6ea309b0a4a44dc9415766897152739"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52994
I20240501 14:09:50.612679 13161 raft_consensus.cc:595] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:50.637564 17126 ts_tablet_manager.cc:902] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:50.637987 17126 tablet_copy_client.cc:308] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:50.638420 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet 73407db0172f454baf2bb44a85add182 from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:36507)
I20240501 14:09:50.638507 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet 73407db0172f454baf2bb44a85add182 from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:36507: session id = 18bef23ddc624bd784ea38fc6ef00040-73407db0172f454baf2bb44a85add182
I20240501 14:09:50.639382 13049 tablet_copy_source_session.cc:205] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:50.640448 17126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73407db0172f454baf2bb44a85add182. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:50.642422 17126 tablet_copy_client.cc:783] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:50.642622 17126 tablet_copy_client.cc:647] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:50.643673 17132 tablet_copy_client.cc:1002] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:50.654613 13290 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "34f677061e42436ab08123227f2be7bf"
dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47728
I20240501 14:09:50.654745 13290 raft_consensus.cc:595] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:50.674468 13161 consensus_queue.cc:237] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:50.675773 12896 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.675949 13028 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:09:50.675982 13096 consensus_peers.cc:485] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bdfa21a72f85470396cc525fba5e379d. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.676119 16052 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.676326 17012 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.677917 17022 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:50.678260 13028 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:50.679459 12896 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:50.680213 12750 catalog_manager.cc:5484] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index -1 to 4, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.687654 13290 consensus_queue.cc:237] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:50.688799 13028 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.688899 13161 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.689186 16641 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 14:09:50.689353 13226 consensus_peers.cc:485] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ab8fdef27b64f87b89f6e8e4166e0c2. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.689455 16071 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.690968 16641 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.691304 13161 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.691470 13028 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.693059 12752 catalog_manager.cc:5484] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.700425 13161 consensus_queue.cc:237] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:50.701610 13028 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:09:50.701650 13290 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:09:50.701841 13095 consensus_peers.cc:485] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9d696c814eaa4c718e49dc32df0d6100. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:50.701925 16052 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.702250 14343 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:09:50.703459 16052 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.703944 13028 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.706027 12750 catalog_manager.cc:5484] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:50.707381 13290 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:50.716941 16071 raft_consensus.cc:1050] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:50.717253 16071 consensus_queue.cc:237] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.718371 13161 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.718370 13028 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.718925 16641 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.719107 14928 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:50.719116 16477 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.719549 16477 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:50.721359 16641 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.721755 14928 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.723655 13028 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.723917 12752 catalog_manager.cc:5484] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.726665 13161 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.728284 13290 consensus_queue.cc:237] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.729224 14928 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.729405 13161 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:50.730125 16641 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.730667 16477 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:50.731631 16477 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.732852 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2a19716d4a9b4acd8a3515b48d9eac32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:50.733333 14928 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.735265 12752 catalog_manager.cc:5484] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.736430 13161 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:50.737551 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a19716d4a9b4acd8a3515b48d9eac32 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:50.737787 16734 tablet_replica.cc:330] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:50.737915 16734 raft_consensus.cc:2229] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:50.738052 16734 raft_consensus.cc:2258] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:50.738952 16734 ts_tablet_manager.cc:1875] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:50.740973 16734 ts_tablet_manager.cc:1888] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:50.741029 16734 log.cc:1196] T 2a19716d4a9b4acd8a3515b48d9eac32 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/2a19716d4a9b4acd8a3515b48d9eac32
I20240501 14:09:50.741736 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 2a19716d4a9b4acd8a3515b48d9eac32 (table rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]) successfully deleted
I20240501 14:09:50.775875 13161 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3"
dest_uuid: "b6ea309b0a4a44dc9415766897152739"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53094
I20240501 14:09:50.775975 13161 raft_consensus.cc:595] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:50.782179 17254 ts_tablet_manager.cc:902] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:50.782642 17254 tablet_copy_client.cc:308] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:50.783044 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 6ac389d29e274b15b5f3db598a04f5e7 from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:59777)
I20240501 14:09:50.783141 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 6ac389d29e274b15b5f3db598a04f5e7 from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:59777: session id = 3025ea3ec516466fbe5c5c4337733584-6ac389d29e274b15b5f3db598a04f5e7
I20240501 14:09:50.783914 13181 tablet_copy_source_session.cc:205] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:50.784760 17254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac389d29e274b15b5f3db598a04f5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:50.786725 17254 tablet_copy_client.cc:783] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:50.786857 17254 tablet_copy_client.cc:647] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:50.787803 17258 tablet_copy_client.cc:1002] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:50.790966 13290 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "34f677061e42436ab08123227f2be7bf"
dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47764
I20240501 14:09:50.791074 13290 raft_consensus.cc:595] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:50.798414 17279 ts_tablet_manager.cc:902] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:50.798928 17279 tablet_copy_client.cc:308] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:50.799394 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 3ca90427253c4320b2fea78e3fc89d92 from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:41969)
I20240501 14:09:50.799490 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 3ca90427253c4320b2fea78e3fc89d92 from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:41969: session id = 70c63e471e5b482b8b257c3e7543feb5-3ca90427253c4320b2fea78e3fc89d92
I20240501 14:09:50.800688 12917 tablet_copy_source_session.cc:205] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:50.801789 17279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca90427253c4320b2fea78e3fc89d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:50.803809 17279 tablet_copy_client.cc:783] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:50.804031 17279 tablet_copy_client.cc:647] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:50.804909 17283 tablet_copy_client.cc:1002] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:50.826186 14343 raft_consensus.cc:979] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: : Instructing follower 70c63e471e5b482b8b257c3e7543feb5 to start an election
I20240501 14:09:50.826272 14343 raft_consensus.cc:1067] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Signalling peer 70c63e471e5b482b8b257c3e7543feb5 to start an election
I20240501 14:09:50.826771 14928 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3"
dest_uuid: "70c63e471e5b482b8b257c3e7543feb5"
from {username='slave'} at 127.6.153.195:45971
I20240501 14:09:50.826933 14928 raft_consensus.cc:482] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:50.827013 14928 raft_consensus.cc:3037] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:50.827709 14928 raft_consensus.cc:504] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.828687 14928 leader_election.cc:288] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [CANDIDATE]: Term 2 election: Requested vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:50.828938 13028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3" candidate_uuid: "70c63e471e5b482b8b257c3e7543feb5" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:50.828981 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3" candidate_uuid: "70c63e471e5b482b8b257c3e7543feb5" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:50.829002 13290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439e22f89e944051b44d3263c43b2fc3" candidate_uuid: "70c63e471e5b482b8b257c3e7543feb5" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:50.829058 13028 raft_consensus.cc:3037] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:50.829068 13161 raft_consensus.cc:3032] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:50.829113 13161 raft_consensus.cc:729] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:50.829114 13290 raft_consensus.cc:3037] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:50.829254 13161 consensus_queue.cc:260] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.829443 13161 raft_consensus.cc:3037] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:50.829851 13290 raft_consensus.cc:2454] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70c63e471e5b482b8b257c3e7543feb5 in term 2.
I20240501 14:09:50.829982 13028 raft_consensus.cc:2454] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70c63e471e5b482b8b257c3e7543feb5 in term 2.
I20240501 14:09:50.830261 14863 leader_election.cc:302] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 70c63e471e5b482b8b257c3e7543feb5, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:09:50.830338 13161 raft_consensus.cc:2454] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70c63e471e5b482b8b257c3e7543feb5 in term 2.
I20240501 14:09:50.830483 16461 raft_consensus.cc:2781] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:50.830663 16461 raft_consensus.cc:686] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 LEADER]: Becoming Leader. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Running, Role: LEADER
I20240501 14:09:50.830953 16461 consensus_queue.cc:237] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.833077 12752 catalog_manager.cc:5484] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: term changed from 1 to 2, leader changed from b6ea309b0a4a44dc9415766897152739 (127.6.153.195) to 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199). New cstate: current_term: 2 leader_uuid: "70c63e471e5b482b8b257c3e7543feb5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:50.833272 17316 ts_tablet_manager.cc:902] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:50.833757 17316 tablet_copy_client.cc:308] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:50.834194 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet 920be22f0370418885ce155d9874607a from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:36507)
I20240501 14:09:50.834300 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet 920be22f0370418885ce155d9874607a from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:36507: session id = 18bef23ddc624bd784ea38fc6ef00040-920be22f0370418885ce155d9874607a
I20240501 14:09:50.835155 13049 tablet_copy_source_session.cc:205] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:50.836114 17316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920be22f0370418885ce155d9874607a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:50.837742 17316 tablet_copy_client.cc:783] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:50.837875 17316 tablet_copy_client.cc:647] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:50.838954 17318 tablet_copy_client.cc:1002] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:50.857173 13290 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "34f677061e42436ab08123227f2be7bf"
dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47800
I20240501 14:09:50.857271 13290 raft_consensus.cc:595] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:50.919363 17352 ts_tablet_manager.cc:902] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:50.919876 17352 tablet_copy_client.cc:308] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:50.920418 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 443ae0978f2145658d910e2ad9fa38d0 from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:58931)
I20240501 14:09:50.920527 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 443ae0978f2145658d910e2ad9fa38d0 from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:58931: session id = 2986bfcc5da34cbca2cad1643b6e35c9-443ae0978f2145658d910e2ad9fa38d0
I20240501 14:09:50.920547 16641 raft_consensus.cc:979] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: : Instructing follower a3d67dc22c8a4cc2ba08b836537105e7 to start an election
I20240501 14:09:50.920624 16641 raft_consensus.cc:1067] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Signalling peer a3d67dc22c8a4cc2ba08b836537105e7 to start an election
I20240501 14:09:50.921156 12896 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "34f677061e42436ab08123227f2be7bf"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
from {username='slave'} at 127.6.153.196:45239
I20240501 14:09:50.921291 12896 raft_consensus.cc:482] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:50.921351 12896 raft_consensus.cc:3037] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:50.921367 13181 tablet_copy_source_session.cc:205] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:50.922176 17352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 443ae0978f2145658d910e2ad9fa38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:50.922230 12896 raft_consensus.cc:504] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.922681 12896 leader_election.cc:288] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 2 election: Requested vote from peers 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347)
I20240501 14:09:50.922828 13292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34f677061e42436ab08123227f2be7bf" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:50.922963 13292 raft_consensus.cc:3032] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:50.923018 13292 raft_consensus.cc:729] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:09:50.923148 13292 consensus_queue.cc:260] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.923316 13292 raft_consensus.cc:3037] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:50.924211 13292 raft_consensus.cc:2454] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 2.
I20240501 14:09:50.924280 17352 tablet_copy_client.cc:783] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:50.924397 17352 tablet_copy_client.cc:647] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:50.924571 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34f677061e42436ab08123227f2be7bf" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:50.924682 13029 raft_consensus.cc:3037] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:50.924839 14928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34f677061e42436ab08123227f2be7bf" candidate_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "70c63e471e5b482b8b257c3e7543feb5"
I20240501 14:09:50.924983 14928 raft_consensus.cc:3037] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:50.925566 13029 raft_consensus.cc:2454] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 2.
I20240501 14:09:50.925917 14928 raft_consensus.cc:2454] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d67dc22c8a4cc2ba08b836537105e7 in term 2.
I20240501 14:09:50.926293 12833 leader_election.cc:302] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3021737e50984ca4bb2ad151dc4776ff, 70c63e471e5b482b8b257c3e7543feb5, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:50.926656 17356 tablet_copy_client.cc:1002] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:50.926846 16692 raft_consensus.cc:2781] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:50.926923 16692 raft_consensus.cc:686] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 LEADER]: Becoming Leader. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:50.927040 16692 consensus_queue.cc:237] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:50.929539 12752 catalog_manager.cc:5484] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: term changed from 1 to 2, leader changed from 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) to a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193). New cstate: current_term: 2 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:51.050257 17359 ts_tablet_manager.cc:902] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:51.050701 17359 tablet_copy_client.cc:308] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:51.051119 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 2f58cd9e96244a2da7954bdaf9272b82 from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:41969)
I20240501 14:09:51.051204 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 2f58cd9e96244a2da7954bdaf9272b82 from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:41969: session id = 70c63e471e5b482b8b257c3e7543feb5-2f58cd9e96244a2da7954bdaf9272b82
I20240501 14:09:51.051935 12917 tablet_copy_source_session.cc:205] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:51.052664 17359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f58cd9e96244a2da7954bdaf9272b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:51.054191 17359 tablet_copy_client.cc:783] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:51.054352 17359 tablet_copy_client.cc:647] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:51.055155 17363 tablet_copy_client.cc:1002] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:51.119596 17409 ts_tablet_manager.cc:902] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:51.120133 17409 tablet_copy_client.cc:308] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:51.120527 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet bdfa21a72f85470396cc525fba5e379d from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:59777)
I20240501 14:09:51.120622 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet bdfa21a72f85470396cc525fba5e379d from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:59777: session id = 3025ea3ec516466fbe5c5c4337733584-bdfa21a72f85470396cc525fba5e379d
I20240501 14:09:51.121249 13181 tablet_copy_source_session.cc:205] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:51.122033 17409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdfa21a72f85470396cc525fba5e379d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:51.124339 17409 tablet_copy_client.cc:783] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:51.124478 17409 tablet_copy_client.cc:647] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:51.125459 17418 tablet_copy_client.cc:1002] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:51.246160 17429 ts_tablet_manager.cc:902] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:51.246618 17429 tablet_copy_client.cc:308] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:51.247246 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 9d696c814eaa4c718e49dc32df0d6100 from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:58931)
I20240501 14:09:51.247371 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 9d696c814eaa4c718e49dc32df0d6100 from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:58931: session id = 2986bfcc5da34cbca2cad1643b6e35c9-9d696c814eaa4c718e49dc32df0d6100
I20240501 14:09:51.248082 13181 tablet_copy_source_session.cc:205] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:51.248889 17429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d696c814eaa4c718e49dc32df0d6100. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:51.250492 17429 tablet_copy_client.cc:783] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:51.250624 17429 tablet_copy_client.cc:647] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:51.251569 17434 tablet_copy_client.cc:1002] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:51.306138 17435 ts_tablet_manager.cc:902] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:51.306581 17435 tablet_copy_client.cc:308] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:51.307001 13312 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:48871)
I20240501 14:09:51.307103 13312 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:48871: session id = 2986bfcc5da34cbca2cad1643b6e35c9-2ab8fdef27b64f87b89f6e8e4166e0c2
I20240501 14:09:51.307894 13312 tablet_copy_source_session.cc:205] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:51.308712 17435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:51.310335 17435 tablet_copy_client.cc:783] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:51.310458 17435 tablet_copy_client.cc:647] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:51.311304 17439 tablet_copy_client.cc:1002] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:51.343958 13029 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 14:09:51.344434 17450 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:51.354900 13161 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 14:09:51.355253 17450 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:51.357905 13292 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 14:09:51.359026 17468 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:51.384302 13292 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 14:09:51.384944 16692 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:51.404673 13029 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 14:09:51.404991 16692 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:51.407027 14928 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 14:09:51.407680 16710 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:51.871101 14928 consensus_queue.cc:237] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:51.872185 13292 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 14:09:51.872275 13029 raft_consensus.cc:1261] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 14:09:51.872555 17468 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:51.872713 17570 consensus_queue.cc:1035] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:51.874226 17468 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:51.875056 13029 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:51.876408 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 439e22f89e944051b44d3263c43b2fc3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:51.876706 12750 catalog_manager.cc:5484] T 439e22f89e944051b44d3263c43b2fc3 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 5 to 7, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 2 leader_uuid: "70c63e471e5b482b8b257c3e7543feb5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:51.876839 13292 raft_consensus.cc:2932] T 439e22f89e944051b44d3263c43b2fc3 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:51.881335 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 439e22f89e944051b44d3263c43b2fc3 with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:51.881636 17585 tablet_replica.cc:330] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:51.882059 17585 raft_consensus.cc:2229] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:51.882236 17585 raft_consensus.cc:2258] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:51.882848 17585 ts_tablet_manager.cc:1875] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:51.885236 17585 ts_tablet_manager.cc:1888] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 14:09:51.885361 17585 log.cc:1196] T 439e22f89e944051b44d3263c43b2fc3 P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/439e22f89e944051b44d3263c43b2fc3
I20240501 14:09:51.885874 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 439e22f89e944051b44d3263c43b2fc3 (table rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]) successfully deleted
I20240501 14:09:51.954788 12896 consensus_queue.cc:237] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:51.955914 14928 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 14:09:51.956069 13029 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 14:09:51.956287 16710 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:51.956480 15214 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:51.958065 15214 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:51.958480 13029 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:51.960615 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 34f677061e42436ab08123227f2be7bf with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:51.961051 14927 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:51.960565 12745 catalog_manager.cc:5484] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 5 to 7, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 2 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:51.963846 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 34f677061e42436ab08123227f2be7bf with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:51.964387 17667 tablet_replica.cc:330] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:51.964728 17667 raft_consensus.cc:2229] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:51.964910 17667 raft_consensus.cc:2258] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:51.965449 17667 ts_tablet_manager.cc:1875] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:51.967537 17667 ts_tablet_manager.cc:1888] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 14:09:51.967612 17667 log.cc:1196] T 34f677061e42436ab08123227f2be7bf P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/34f677061e42436ab08123227f2be7bf
I20240501 14:09:51.968107 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 34f677061e42436ab08123227f2be7bf (table rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]) successfully deleted
I20240501 14:09:51.989761 17023 tablet_copy_client.cc:523] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:51.991062 17023 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:52.011898 17023 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.012146 17023 tablet_bootstrap.cc:492] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:52.012367 17023 ts_tablet_manager.cc:1367] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.021s user 0.022s sys 0.000s
I20240501 14:09:52.012866 17023 raft_consensus.cc:348] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:52.013042 17023 raft_consensus.cc:729] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:52.013177 17023 consensus_queue.cc:260] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:52.013504 17023 ts_tablet_manager.cc:1398] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:52.013945 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-8a948907bfb14d0d958ebb969e5a6a50 received from {username='slave'} at 127.6.153.197:57299
I20240501 14:09:52.014024 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-8a948907bfb14d0d958ebb969e5a6a50 on tablet 8a948907bfb14d0d958ebb969e5a6a50 with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:52.081933 14665 raft_consensus.cc:1203] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:52.143925 17126 tablet_copy_client.cc:523] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.145185 17126 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:52.162643 17126 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.162869 17126 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:52.163012 17126 ts_tablet_manager.cc:1367] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20240501 14:09:52.163431 17126 raft_consensus.cc:348] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:52.163543 17126 raft_consensus.cc:729] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:52.163650 17126 consensus_queue.cc:260] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:52.163982 17126 ts_tablet_manager.cc:1398] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:09:52.164413 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-73407db0172f454baf2bb44a85add182 received from {username='slave'} at 127.6.153.198:36507
I20240501 14:09:52.164495 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-73407db0172f454baf2bb44a85add182 on tablet 73407db0172f454baf2bb44a85add182 with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:52.288074 17254 tablet_copy_client.cc:523] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.289535 17254 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:52.305176 17279 tablet_copy_client.cc:523] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.306354 17279 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:52.306783 17254 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.307031 17254 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:52.307185 17254 ts_tablet_manager.cc:1367] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20240501 14:09:52.307564 17254 raft_consensus.cc:348] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:52.307659 17254 raft_consensus.cc:729] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:52.307735 17254 consensus_queue.cc:260] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:52.308027 17254 ts_tablet_manager.cc:1398] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240501 14:09:52.308516 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-6ac389d29e274b15b5f3db598a04f5e7 received from {username='slave'} at 127.6.153.200:59777
I20240501 14:09:52.308589 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-6ac389d29e274b15b5f3db598a04f5e7 on tablet 6ac389d29e274b15b5f3db598a04f5e7 with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:52.323695 17279 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.323907 17279 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:52.324048 17279 ts_tablet_manager.cc:1367] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.018s user 0.018s sys 0.000s
I20240501 14:09:52.324391 17279 raft_consensus.cc:348] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:52.324486 17279 raft_consensus.cc:729] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:52.324570 17279 consensus_queue.cc:260] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:52.324774 17279 ts_tablet_manager.cc:1398] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:52.325217 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-3ca90427253c4320b2fea78e3fc89d92 received from {username='slave'} at 127.6.153.199:41969
I20240501 14:09:52.325296 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-3ca90427253c4320b2fea78e3fc89d92 on tablet 3ca90427253c4320b2fea78e3fc89d92 with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:52.339219 17316 tablet_copy_client.cc:523] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.340329 17316 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:52.356886 17316 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.357127 17316 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:52.357280 17316 ts_tablet_manager.cc:1367] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.003s
I20240501 14:09:52.357662 17316 raft_consensus.cc:348] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:52.357759 17316 raft_consensus.cc:729] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:52.357838 17316 consensus_queue.cc:260] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:52.358058 17316 ts_tablet_manager.cc:1398] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:52.358529 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-920be22f0370418885ce155d9874607a received from {username='slave'} at 127.6.153.198:36507
I20240501 14:09:52.358616 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-920be22f0370418885ce155d9874607a on tablet 920be22f0370418885ce155d9874607a with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:52.402781 14796 raft_consensus.cc:1203] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:52.426939 17352 tablet_copy_client.cc:523] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.428143 17352 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:52.445060 17352 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.445334 17352 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:52.445495 17352 ts_tablet_manager.cc:1367] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20240501 14:09:52.445840 17352 raft_consensus.cc:348] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:52.445940 17352 raft_consensus.cc:729] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:52.446036 17352 consensus_queue.cc:260] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:52.446249 17352 ts_tablet_manager.cc:1398] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:52.446692 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-443ae0978f2145658d910e2ad9fa38d0 received from {username='slave'} at 127.6.153.197:58931
I20240501 14:09:52.446779 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-443ae0978f2145658d910e2ad9fa38d0 on tablet 443ae0978f2145658d910e2ad9fa38d0 with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:52.510386 15060 raft_consensus.cc:1203] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:52.555428 17359 tablet_copy_client.cc:523] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.556618 17359 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:52.573057 17359 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.573273 17359 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:52.573417 17359 ts_tablet_manager.cc:1367] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20240501 14:09:52.573766 17359 raft_consensus.cc:348] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:52.573863 17359 raft_consensus.cc:729] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:52.573946 17359 consensus_queue.cc:260] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:52.574198 17359 ts_tablet_manager.cc:1398] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:52.574641 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-2f58cd9e96244a2da7954bdaf9272b82 received from {username='slave'} at 127.6.153.199:41969
I20240501 14:09:52.574715 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-2f58cd9e96244a2da7954bdaf9272b82 on tablet 2f58cd9e96244a2da7954bdaf9272b82 with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:52.597486 14796 raft_consensus.cc:1203] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:52.611876 14927 raft_consensus.cc:1203] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:52.615088 17854 raft_consensus.cc:1050] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:52.615511 17854 consensus_queue.cc:237] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:52.616763 13029 raft_consensus.cc:1261] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:52.616778 13161 raft_consensus.cc:1261] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:52.616778 14665 raft_consensus.cc:1261] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:52.617174 16710 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:52.617360 17854 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:52.617623 17856 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:52.619480 16710 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:52.619879 14665 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:52.619953 13161 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:52.621891 13029 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:52.622282 12750 catalog_manager.cc:5484] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:52.625170 12896 consensus_queue.cc:237] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:52.625722 17409 tablet_copy_client.cc:523] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.626206 13029 raft_consensus.cc:1261] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:52.626317 14665 raft_consensus.cc:1261] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:52.626621 17856 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:52.626766 15214 consensus_queue.cc:1035] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:52.627099 17409 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:52.628111 15214 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:52.630069 13029 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:52.630257 12748 catalog_manager.cc:5484] T 8a948907bfb14d0d958ebb969e5a6a50 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:52.630934 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8a948907bfb14d0d958ebb969e5a6a50 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:52.632539 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a948907bfb14d0d958ebb969e5a6a50 with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:52.639732 17864 tablet_replica.cc:330] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:52.639899 17864 raft_consensus.cc:2229] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:52.640303 14665 raft_consensus.cc:2932] T 8a948907bfb14d0d958ebb969e5a6a50 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:52.640517 17864 raft_consensus.cc:2258] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:52.642441 17864 ts_tablet_manager.cc:1875] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:52.644544 17864 ts_tablet_manager.cc:1888] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:52.644618 17864 log.cc:1196] T 8a948907bfb14d0d958ebb969e5a6a50 P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/8a948907bfb14d0d958ebb969e5a6a50
I20240501 14:09:52.645082 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 8a948907bfb14d0d958ebb969e5a6a50 (table rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]) successfully deleted
I20240501 14:09:52.648870 17409 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.649070 17409 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:52.649184 17409 ts_tablet_manager.cc:1367] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.001s
I20240501 14:09:52.649502 17409 raft_consensus.cc:348] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:52.649590 17409 raft_consensus.cc:729] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:52.649662 17409 consensus_queue.cc:260] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:52.649878 17409 ts_tablet_manager.cc:1398] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:52.650298 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-bdfa21a72f85470396cc525fba5e379d received from {username='slave'} at 127.6.153.200:59777
I20240501 14:09:52.650369 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-bdfa21a72f85470396cc525fba5e379d on tablet bdfa21a72f85470396cc525fba5e379d with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:52.751855 17429 tablet_copy_client.cc:523] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.753337 17429 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:52.756461 14927 raft_consensus.cc:1203] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:52.774128 17429 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.774350 17429 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:52.774497 17429 ts_tablet_manager.cc:1367] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.021s user 0.012s sys 0.005s
I20240501 14:09:52.774829 17429 raft_consensus.cc:348] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:52.774930 17429 raft_consensus.cc:729] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:52.775013 17429 consensus_queue.cc:260] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:52.775277 17429 ts_tablet_manager.cc:1398] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:52.775913 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-9d696c814eaa4c718e49dc32df0d6100 received from {username='slave'} at 127.6.153.197:58931
I20240501 14:09:52.775990 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-9d696c814eaa4c718e49dc32df0d6100 on tablet 9d696c814eaa4c718e49dc32df0d6100 with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:52.811585 17435 tablet_copy_client.cc:523] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:52.812969 17435 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:52.833065 17435 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:52.833302 17435 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:52.833467 17435 ts_tablet_manager.cc:1367] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.021s user 0.020s sys 0.000s
I20240501 14:09:52.833886 17435 raft_consensus.cc:348] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:52.834009 17435 raft_consensus.cc:729] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:52.834111 17435 consensus_queue.cc:260] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:52.834367 17435 ts_tablet_manager.cc:1398] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:52.834826 13312 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-2ab8fdef27b64f87b89f6e8e4166e0c2 received from {username='slave'} at 127.6.153.197:48871
I20240501 14:09:52.834894 13312 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-2ab8fdef27b64f87b89f6e8e4166e0c2 on tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:52.898221 17964 raft_consensus.cc:1050] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:52.898566 17964 consensus_queue.cc:237] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:52.899863 12896 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:52.899868 13292 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:52.900012 15060 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:52.900243 17964 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:52.900422 17963 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:52.900609 14343 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:52.902583 17964 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:52.902940 15060 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:52.904281 12896 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:52.906080 12752 catalog_manager.cc:5484] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:52.906769 13292 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:52.910383 13161 consensus_queue.cc:237] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:52.911273 15060 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:52.911399 13292 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:52.911695 17962 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:52.911868 17963 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:52.913426 17964 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:52.913856 15060 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:52.914501 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6ac389d29e274b15b5f3db598a04f5e7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:52.915725 12748 catalog_manager.cc:5484] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:52.916283 13292 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:52.918267 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac389d29e274b15b5f3db598a04f5e7 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:52.918742 17969 tablet_replica.cc:330] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:52.918913 17969 raft_consensus.cc:2229] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:52.919114 17969 raft_consensus.cc:2258] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:52.919863 17969 ts_tablet_manager.cc:1875] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:52.921561 17969 ts_tablet_manager.cc:1888] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:52.921633 17969 log.cc:1196] T 6ac389d29e274b15b5f3db598a04f5e7 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/6ac389d29e274b15b5f3db598a04f5e7
I20240501 14:09:52.922180 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 6ac389d29e274b15b5f3db598a04f5e7 (table rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]) successfully deleted
I20240501 14:09:52.927812 14665 raft_consensus.cc:1203] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:52.999106 17970 raft_consensus.cc:1050] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:52.999575 17970 consensus_queue.cc:237] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:53.000828 14796 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.000823 13161 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.000917 13292 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.001228 17973 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.001420 17971 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.001631 17970 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.003221 17971 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.003629 13161 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.003806 13292 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.005323 14796 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.006842 12748 catalog_manager.cc:5484] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.010339 13029 consensus_queue.cc:237] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:53.011233 14796 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.011358 13161 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.011598 17971 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.011755 17973 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.013020 17971 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.013387 14796 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.014323 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 920be22f0370418885ce155d9874607a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:53.015646 12750 catalog_manager.cc:5484] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.016906 13161 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.017786 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 920be22f0370418885ce155d9874607a with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:53.018121 17977 tablet_replica.cc:330] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:53.018441 17977 raft_consensus.cc:2229] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:53.018610 17977 raft_consensus.cc:2258] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:53.019196 17977 ts_tablet_manager.cc:1875] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:53.020984 17977 ts_tablet_manager.cc:1888] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:53.021054 17977 log.cc:1196] T 920be22f0370418885ce155d9874607a P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/920be22f0370418885ce155d9874607a
I20240501 14:09:53.021293 15060 raft_consensus.cc:1203] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:53.021590 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 920be22f0370418885ce155d9874607a (table rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]) successfully deleted
I20240501 14:09:53.112396 17971 raft_consensus.cc:1050] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:53.112804 17971 consensus_queue.cc:237] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:53.114113 14796 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.114473 12896 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.114471 17357 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.114773 13161 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.114925 17357 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.116093 17357 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.117849 17357 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.119518 13161 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.120141 14796 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.120188 12896 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.120591 12750 catalog_manager.cc:5484] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 4 to 5, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:53.124109 13029 consensus_queue.cc:237] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:53.125113 12896 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.125138 14796 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.125483 17970 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.125650 17971 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.127025 17970 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.127432 14796 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.127583 12896 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.129345 12745 catalog_manager.cc:5484] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: config changed from index 5 to 6, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:53.129516 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 73407db0172f454baf2bb44a85add182 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:53.131994 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 73407db0172f454baf2bb44a85add182 with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:53.132170 17864 tablet_replica.cc:330] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:53.132289 17864 raft_consensus.cc:2229] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:53.132418 17864 raft_consensus.cc:2258] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:53.133306 17864 ts_tablet_manager.cc:1875] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:53.134987 17864 ts_tablet_manager.cc:1888] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:53.135046 17864 log.cc:1196] T 73407db0172f454baf2bb44a85add182 P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/73407db0172f454baf2bb44a85add182
I20240501 14:09:53.135452 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 73407db0172f454baf2bb44a85add182 (table rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]) successfully deleted
I20240501 14:09:53.175289 16710 raft_consensus.cc:1050] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:53.175648 16710 consensus_queue.cc:237] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:53.176779 13161 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.176887 14927 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.176949 13292 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.177263 18094 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.177446 17856 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.177603 17854 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.179443 17854 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.179983 14927 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.181135 13161 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.182202 12748 catalog_manager.cc:5484] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.182687 13292 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.185837 12896 consensus_queue.cc:237] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:53.186841 13161 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.186908 14927 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.187179 18094 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.187356 16710 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.188630 16710 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.189023 14927 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.189325 13161 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.190512 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2f58cd9e96244a2da7954bdaf9272b82 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:53.191023 12752 catalog_manager.cc:5484] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 5 to 6, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.193567 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f58cd9e96244a2da7954bdaf9272b82 with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:53.193737 17977 tablet_replica.cc:330] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:53.193853 17977 raft_consensus.cc:2229] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:53.193996 17977 raft_consensus.cc:2258] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:53.194717 17977 ts_tablet_manager.cc:1875] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:53.196420 17977 ts_tablet_manager.cc:1888] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:53.196482 17977 log.cc:1196] T 2f58cd9e96244a2da7954bdaf9272b82 P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/2f58cd9e96244a2da7954bdaf9272b82
I20240501 14:09:53.196838 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 2f58cd9e96244a2da7954bdaf9272b82 (table rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]) successfully deleted
I20240501 14:09:53.223487 17854 raft_consensus.cc:1050] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:53.223832 17854 consensus_queue.cc:237] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:53.224952 14927 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.225023 13292 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.225097 13029 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.225340 17854 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.225517 16710 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.225713 18094 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.227455 16710 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.227842 14927 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.227934 13029 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.229285 13292 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.230347 12750 catalog_manager.cc:5484] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 4 to 5, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.231132 14665 raft_consensus.cc:1203] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:53.233637 12896 consensus_queue.cc:237] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:53.234674 14927 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.234763 13292 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.235016 18094 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.235183 17856 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.236588 17854 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.236974 13292 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.237022 14927 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.238209 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3ca90427253c4320b2fea78e3fc89d92 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:53.239187 12748 catalog_manager.cc:5484] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:53.241014 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ca90427253c4320b2fea78e3fc89d92 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:53.241392 18102 tablet_replica.cc:330] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:53.241504 18102 raft_consensus.cc:2229] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:53.241639 18102 raft_consensus.cc:2258] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:53.242189 18102 ts_tablet_manager.cc:1875] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:53.243865 18102 ts_tablet_manager.cc:1888] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:53.243925 18102 log.cc:1196] T 3ca90427253c4320b2fea78e3fc89d92 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/3ca90427253c4320b2fea78e3fc89d92
I20240501 14:09:53.244307 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 3ca90427253c4320b2fea78e3fc89d92 (table rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]) successfully deleted
I20240501 14:09:53.286047 14665 raft_consensus.cc:1203] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:09:53.399586 17964 raft_consensus.cc:1050] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:53.399942 17964 consensus_queue.cc:237] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:53.401144 12895 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.401234 14665 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.401293 13292 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.401881 17964 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.402128 14343 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.402640 14343 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.404469 14343 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.404888 13292 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.406867 12895 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.407475 12752 catalog_manager.cc:5484] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.412174 14665 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.414521 13161 consensus_queue.cc:237] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:53.415766 13292 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.415766 14665 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.416163 17963 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.416343 17962 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.417929 14343 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.418306 13292 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.419082 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 443ae0978f2145658d910e2ad9fa38d0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:53.420306 14665 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.420368 12745 catalog_manager.cc:5484] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.423063 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 443ae0978f2145658d910e2ad9fa38d0 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:53.423359 18158 tablet_replica.cc:330] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:53.423496 18158 raft_consensus.cc:2229] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:53.423647 18158 raft_consensus.cc:2258] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:53.424321 18158 ts_tablet_manager.cc:1875] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:53.426612 18158 ts_tablet_manager.cc:1888] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:53.426689 18158 log.cc:1196] T 443ae0978f2145658d910e2ad9fa38d0 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/443ae0978f2145658d910e2ad9fa38d0
I20240501 14:09:53.427147 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 443ae0978f2145658d910e2ad9fa38d0 (table rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]) successfully deleted
I20240501 14:09:53.512378 14343 raft_consensus.cc:1050] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:53.512851 14343 consensus_queue.cc:237] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:53.514173 15060 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.514276 13029 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.514300 12895 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.514596 17964 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.514756 17962 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.514940 14343 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.517534 17962 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:53.518030 12895 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:53.520104 15060 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:53.520398 12752 catalog_manager.cc:5484] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:53.522025 13029 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:53.524863 13161 consensus_queue.cc:237] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:53.525818 15060 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.525866 13029 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.526154 17963 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.526427 17962 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.527493 17962 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:53.527822 15060 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:53.528306 13029 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:53.528528 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bdfa21a72f85470396cc525fba5e379d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:53.529618 12748 catalog_manager.cc:5484] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 5 to 6, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:53.531888 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet bdfa21a72f85470396cc525fba5e379d with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:53.532069 18158 tablet_replica.cc:330] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:53.532202 18158 raft_consensus.cc:2229] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:53.532353 18158 raft_consensus.cc:2258] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:53.532918 18158 ts_tablet_manager.cc:1875] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:53.534794 18158 ts_tablet_manager.cc:1888] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:53.534852 18158 log.cc:1196] T bdfa21a72f85470396cc525fba5e379d P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/bdfa21a72f85470396cc525fba5e379d
I20240501 14:09:53.535444 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet bdfa21a72f85470396cc525fba5e379d (table rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]) successfully deleted
I20240501 14:09:53.729938 17962 raft_consensus.cc:1050] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:53.730403 17962 consensus_queue.cc:237] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:53.731763 14665 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.731781 13029 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.731787 13292 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.732167 17964 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.732350 17962 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.732524 14343 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.734607 14343 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.735024 13029 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.735231 14665 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.737341 13292 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.737648 12745 catalog_manager.cc:5484] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.740783 13161 consensus_queue.cc:237] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:53.741868 13292 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.742203 17964 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.742302 14665 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.742599 17964 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.743585 17962 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.744024 13292 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.744645 14665 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.744937 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9d696c814eaa4c718e49dc32df0d6100 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:53.746730 12747 catalog_manager.cc:5484] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:53.748836 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d696c814eaa4c718e49dc32df0d6100 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:53.749161 18208 tablet_replica.cc:330] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:53.749321 18208 raft_consensus.cc:2229] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:53.749471 18208 raft_consensus.cc:2258] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:53.749981 18208 ts_tablet_manager.cc:1875] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:53.751659 18208 ts_tablet_manager.cc:1888] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:53.751721 18208 log.cc:1196] T 9d696c814eaa4c718e49dc32df0d6100 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/9d696c814eaa4c718e49dc32df0d6100
I20240501 14:09:53.752125 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 9d696c814eaa4c718e49dc32df0d6100 (table rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]) successfully deleted
I20240501 14:09:53.851375 18209 raft_consensus.cc:1050] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:53.851749 18209 consensus_queue.cc:237] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:53.853266 13029 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.853296 14665 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.853343 13161 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:09:53.853688 18211 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.853885 17569 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.854024 18207 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:09:53.855784 17569 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.856300 13161 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.856300 14665 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.858630 12748 catalog_manager.cc:5484] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:53.859758 13029 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.861913 13292 consensus_queue.cc:237] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:53.863123 14665 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.863204 13161 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:09:53.863513 18211 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.863732 18207 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:09:53.865159 18209 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.865571 13161 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.865612 14665 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:53.866408 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:53.867482 12747 catalog_manager.cc:5484] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 5 to 6, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:53.870199 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:53.870371 18208 tablet_replica.cc:330] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:53.870496 18208 raft_consensus.cc:2229] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:53.870657 18208 raft_consensus.cc:2258] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:53.871312 18208 ts_tablet_manager.cc:1875] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:53.873239 18208 ts_tablet_manager.cc:1888] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:09:53.873301 18208 log.cc:1196] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/2ab8fdef27b64f87b89f6e8e4166e0c2
I20240501 14:09:53.873695 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 (table rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]) successfully deleted
I20240501 14:09:53.945050 13271 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:53.948848 14908 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:09:53.953012 14766 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:53.963670 13007 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:09:53.965848 15040 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:53.966549 15983 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:53.967010 14644 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:09:53.968951 12876 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 14:09:53.969448 13141 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 14:09:54.023880 13292 consensus_queue.cc:237] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:54.025099 13161 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.025128 14665 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.025449 18211 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.025652 18207 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.027268 18207 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.027683 13161 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.029917 14665 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.030150 12749 catalog_manager.cc:5484] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 14:09:54.031494 13227 consensus_peers.cc:485] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ab8fdef27b64f87b89f6e8e4166e0c2. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.038707 13161 consensus_queue.cc:237] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:54.039819 14665 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.039849 13292 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:54.040038 13094 consensus_peers.cc:485] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9d696c814eaa4c718e49dc32df0d6100. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.040145 17963 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.040298 17964 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.041592 17964 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.042125 14665 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.043076 13292 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.044194 12747 catalog_manager.cc:5484] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 reported cstate change: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.051551 12895 consensus_queue.cc:237] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:54.052742 13029 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240501 14:09:54.052768 14927 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
W20240501 14:09:54.053009 12832 consensus_peers.cc:485] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34f677061e42436ab08123227f2be7bf. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.053207 18290 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:54.053421 17856 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:54.055313 18094 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.055724 14927 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.055931 13029 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.058353 12746 catalog_manager.cc:5484] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 7 to 8, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 2 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.065228 12895 consensus_queue.cc:237] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:54.066210 14927 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.066360 13160 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.066614 17856 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240501 14:09:54.066756 12831 consensus_peers.cc:485] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879): Couldn't send request to peer 2986bfcc5da34cbca2cad1643b6e35c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f58cd9e96244a2da7954bdaf9272b82. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.066776 18293 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.068503 18290 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:54.068885 14927 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:54.070614 13160 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } } }
I20240501 14:09:54.071002 12751 catalog_manager.cc:5484] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 6 to 7, NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:54.079464 13160 consensus_queue.cc:237] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:54.080669 15060 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:54.080690 13097 consensus_peers.cc:485] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e34fd488c214188aac9990af9de0db4. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.081073 17964 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.081122 13292 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.081454 17964 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.082794 14343 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.083146 15060 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.083451 13292 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.085847 12747 catalog_manager.cc:5484] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.092619 12895 consensus_queue.cc:237] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:54.093808 14927 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.093974 13292 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.094347 18094 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.094532 18293 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.096071 18293 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.096447 14927 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.096501 13292 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
W20240501 14:09:54.098228 12832 consensus_peers.cc:485] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ca90427253c4320b2fea78e3fc89d92. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.098721 12751 catalog_manager.cc:5484] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.105297 12895 consensus_queue.cc:237] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:54.106396 13292 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.106536 14927 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:54.106624 12830 consensus_peers.cc:485] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37047d45ffe14570b28e459afdb66d7e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.106787 17856 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.106958 18094 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.108493 18290 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.108913 14927 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.108944 13292 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.110980 12746 catalog_manager.cc:5484] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.117581 13029 consensus_queue.cc:237] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:54.118716 12895 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.118784 14796 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:54.118994 12965 consensus_peers.cc:485] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73407db0172f454baf2bb44a85add182. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.119181 18328 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.119361 18327 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.120888 17973 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.121284 14796 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.121311 12895 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.123680 12747 catalog_manager.cc:5484] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.129734 12895 consensus_queue.cc:237] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:54.130957 14796 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.130993 13160 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:54.130995 12832 consensus_peers.cc:485] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 627fd32202d247c2af434ab8be152102. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.131275 18293 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.131461 18094 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.132925 17856 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.133360 14796 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.134268 13160 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.135490 12747 catalog_manager.cc:5484] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: config changed from index 6 to 7, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.141439 12895 consensus_queue.cc:237] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:54.142554 14796 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.142640 13160 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:54.142796 12832 consensus_peers.cc:485] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fc05e3adc8e4ed0b45d275725f289f4. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.142948 18293 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.143262 18290 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.144464 18290 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.144888 14796 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.145200 13160 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.146883 12751 catalog_manager.cc:5484] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:54.153898 13160 consensus_queue.cc:237] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:54.154994 13292 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.155144 14665 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.155629 17962 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.155922 17963 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.157193 14343 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
W20240501 14:09:54.158185 13096 consensus_peers.cc:485] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 443ae0978f2145658d910e2ad9fa38d0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.158605 14665 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.159300 12747 catalog_manager.cc:5484] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:54.160212 13292 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.166647 13292 consensus_queue.cc:237] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:54.167981 14927 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.168478 18211 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240501 14:09:54.168644 13225 consensus_peers.cc:485] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e -> Peer 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509): Couldn't send request to peer 18bef23ddc624bd784ea38fc6ef00040. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a19716d4a9b4acd8a3515b48d9eac32. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.168807 13160 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.169562 18211 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.170039 17569 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.170466 14927 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.172351 13160 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } } }
I20240501 14:09:54.174266 12749 catalog_manager.cc:5484] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 6 to 7, NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:54.178781 12895 consensus_queue.cc:237] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:54.180189 14665 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.180666 18293 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.182385 13029 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.182598 18094 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
W20240501 14:09:54.182761 12832 consensus_peers.cc:485] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe046c56dde144f9adbbbf779067476e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.183069 14665 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.184888 12751 catalog_manager.cc:5484] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:54.184971 17856 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.187155 13029 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:54.193105 13029 consensus_queue.cc:237] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:54.194368 13160 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.194389 14796 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:54.194412 12965 consensus_peers.cc:485] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff -> Peer 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347): Couldn't send request to peer 70c63e471e5b482b8b257c3e7543feb5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 920be22f0370418885ce155d9874607a. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.194706 17973 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.194881 18328 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.196456 17973 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.196882 13160 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.197129 14796 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } } }
I20240501 14:09:54.198729 12751 catalog_manager.cc:5484] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 6 to 7, NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:54.205754 13029 consensus_queue.cc:237] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:54.206871 12895 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:54.206938 14927 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:54.207147 12964 consensus_peers.cc:485] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff -> Peer 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743): Couldn't send request to peer 3025ea3ec516466fbe5c5c4337733584. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5053c0f554af4efaa221444af87dd9b9. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:54.207273 18327 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.207448 17973 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:54.208992 18327 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.209632 14927 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.209632 12895 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.212144 12749 catalog_manager.cc:5484] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 6 to 7, NON_VOTER 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } } }
I20240501 14:09:54.429132 18460 ts_tablet_manager.cc:902] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:54.429582 18460 tablet_copy_client.cc:308] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:54.430035 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 9d696c814eaa4c718e49dc32df0d6100 from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:59809)
I20240501 14:09:54.430157 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 9d696c814eaa4c718e49dc32df0d6100 from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:59809: session id = 18bef23ddc624bd784ea38fc6ef00040-9d696c814eaa4c718e49dc32df0d6100
I20240501 14:09:54.430866 13181 tablet_copy_source_session.cc:205] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.431674 18460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d696c814eaa4c718e49dc32df0d6100. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.433409 18460 tablet_copy_client.cc:783] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.433545 18460 tablet_copy_client.cc:647] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.434432 18464 tablet_copy_client.cc:1002] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.463682 18465 ts_tablet_manager.cc:902] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:54.464129 18465 tablet_copy_client.cc:308] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:54.464551 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 7e34fd488c214188aac9990af9de0db4 from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:52497)
I20240501 14:09:54.464650 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 7e34fd488c214188aac9990af9de0db4 from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:52497: session id = 70c63e471e5b482b8b257c3e7543feb5-7e34fd488c214188aac9990af9de0db4
I20240501 14:09:54.465288 13181 tablet_copy_source_session.cc:205] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.466008 18465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e34fd488c214188aac9990af9de0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.467743 18465 tablet_copy_client.cc:783] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.467883 18465 tablet_copy_client.cc:647] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.468698 18469 tablet_copy_client.cc:1002] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.481767 18470 ts_tablet_manager.cc:902] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:54.482223 18470 tablet_copy_client.cc:308] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:54.482642 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 34f677061e42436ab08123227f2be7bf from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:55411)
I20240501 14:09:54.482750 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 34f677061e42436ab08123227f2be7bf from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:55411: session id = 3025ea3ec516466fbe5c5c4337733584-34f677061e42436ab08123227f2be7bf
I20240501 14:09:54.483500 12917 tablet_copy_source_session.cc:205] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.484287 18470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34f677061e42436ab08123227f2be7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.486315 18470 tablet_copy_client.cc:783] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.486454 18470 tablet_copy_client.cc:647] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.487416 18474 tablet_copy_client.cc:1002] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.494112 18485 ts_tablet_manager.cc:902] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:54.494843 18485 tablet_copy_client.cc:308] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:54.498144 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 3ca90427253c4320b2fea78e3fc89d92 from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:40601)
I20240501 14:09:54.498222 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 3ca90427253c4320b2fea78e3fc89d92 from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:40601: session id = 777595e1371c421fb350f0bfbb7a8035-3ca90427253c4320b2fea78e3fc89d92
I20240501 14:09:54.498881 12917 tablet_copy_source_session.cc:205] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.502065 18485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca90427253c4320b2fea78e3fc89d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.504796 18501 ts_tablet_manager.cc:902] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:54.505261 18501 tablet_copy_client.cc:308] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:54.507655 18485 tablet_copy_client.cc:783] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.507851 18485 tablet_copy_client.cc:647] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.509155 18512 ts_tablet_manager.cc:902] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:54.509630 18512 tablet_copy_client.cc:308] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:54.509999 18494 tablet_copy_client.cc:1002] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.510429 13312 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:45235)
I20240501 14:09:54.510514 13312 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:45235: session id = 777595e1371c421fb350f0bfbb7a8035-2ab8fdef27b64f87b89f6e8e4166e0c2
I20240501 14:09:54.511237 13312 tablet_copy_source_session.cc:205] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.511677 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 2f58cd9e96244a2da7954bdaf9272b82 from peer 2986bfcc5da34cbca2cad1643b6e35c9 ({username='slave'} at 127.6.153.197:57299)
I20240501 14:09:54.511766 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 2f58cd9e96244a2da7954bdaf9272b82 from peer 2986bfcc5da34cbca2cad1643b6e35c9 at {username='slave'} at 127.6.153.197:57299: session id = 2986bfcc5da34cbca2cad1643b6e35c9-2f58cd9e96244a2da7954bdaf9272b82
I20240501 14:09:54.511937 18501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.512475 12917 tablet_copy_source_session.cc:205] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.513669 18501 tablet_copy_client.cc:783] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.513790 18501 tablet_copy_client.cc:647] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.514477 18512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f58cd9e96244a2da7954bdaf9272b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.514837 18503 tablet_copy_client.cc:1002] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.515753 18522 ts_tablet_manager.cc:902] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:54.516366 18522 tablet_copy_client.cc:308] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:54.516829 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 37047d45ffe14570b28e459afdb66d7e from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:58259)
I20240501 14:09:54.516875 18512 tablet_copy_client.cc:783] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.516912 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 37047d45ffe14570b28e459afdb66d7e from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:58259: session id = 18bef23ddc624bd784ea38fc6ef00040-37047d45ffe14570b28e459afdb66d7e
I20240501 14:09:54.517014 18512 tablet_copy_client.cc:647] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.517622 12917 tablet_copy_source_session.cc:205] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.518431 18515 tablet_copy_client.cc:1002] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.519538 18522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37047d45ffe14570b28e459afdb66d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.521613 18522 tablet_copy_client.cc:783] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.521745 18522 tablet_copy_client.cc:647] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.522594 18526 tablet_copy_client.cc:1002] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.542867 18552 ts_tablet_manager.cc:902] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:54.543349 18552 tablet_copy_client.cc:308] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:54.543713 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 627fd32202d247c2af434ab8be152102 from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:55411)
I20240501 14:09:54.543798 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 627fd32202d247c2af434ab8be152102 from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:55411: session id = 3025ea3ec516466fbe5c5c4337733584-627fd32202d247c2af434ab8be152102
I20240501 14:09:54.544543 12917 tablet_copy_source_session.cc:205] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.545513 18552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627fd32202d247c2af434ab8be152102. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.547550 18552 tablet_copy_client.cc:783] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.547735 18552 tablet_copy_client.cc:647] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.548616 18556 tablet_copy_client.cc:1002] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.604375 18577 ts_tablet_manager.cc:902] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:54.604820 18577 tablet_copy_client.cc:308] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:54.605226 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet 920be22f0370418885ce155d9874607a from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:52399)
I20240501 14:09:54.605324 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet 920be22f0370418885ce155d9874607a from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:52399: session id = 70c63e471e5b482b8b257c3e7543feb5-920be22f0370418885ce155d9874607a
I20240501 14:09:54.605978 13049 tablet_copy_source_session.cc:205] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.606717 18577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920be22f0370418885ce155d9874607a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.608424 18577 tablet_copy_client.cc:783] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.608657 18577 tablet_copy_client.cc:647] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.609577 18581 tablet_copy_client.cc:1002] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.616336 18582 ts_tablet_manager.cc:902] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:54.616751 18582 tablet_copy_client.cc:308] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:54.617105 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet 73407db0172f454baf2bb44a85add182 from peer 70c63e471e5b482b8b257c3e7543feb5 ({username='slave'} at 127.6.153.199:52399)
I20240501 14:09:54.617182 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet 73407db0172f454baf2bb44a85add182 from peer 70c63e471e5b482b8b257c3e7543feb5 at {username='slave'} at 127.6.153.199:52399: session id = 70c63e471e5b482b8b257c3e7543feb5-73407db0172f454baf2bb44a85add182
I20240501 14:09:54.617772 13049 tablet_copy_source_session.cc:205] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.618410 18582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73407db0172f454baf2bb44a85add182. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.619879 18582 tablet_copy_client.cc:783] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.620000 18582 tablet_copy_client.cc:647] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.620874 18586 tablet_copy_client.cc:1002] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.646835 18587 ts_tablet_manager.cc:902] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:54.647292 18587 tablet_copy_client.cc:308] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:54.650144 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 443ae0978f2145658d910e2ad9fa38d0 from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:36143)
I20240501 14:09:54.650241 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 443ae0978f2145658d910e2ad9fa38d0 from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:36143: session id = 777595e1371c421fb350f0bfbb7a8035-443ae0978f2145658d910e2ad9fa38d0
I20240501 14:09:54.650972 13181 tablet_copy_source_session.cc:205] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.651706 18587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 443ae0978f2145658d910e2ad9fa38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.653350 18587 tablet_copy_client.cc:783] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.653468 18587 tablet_copy_client.cc:647] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.654301 18591 tablet_copy_client.cc:1002] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.695119 18592 ts_tablet_manager.cc:902] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:54.695590 18592 tablet_copy_client.cc:308] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:54.695995 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet fe046c56dde144f9adbbbf779067476e from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:40601)
I20240501 14:09:54.696097 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet fe046c56dde144f9adbbbf779067476e from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:40601: session id = 777595e1371c421fb350f0bfbb7a8035-fe046c56dde144f9adbbbf779067476e
I20240501 14:09:54.696755 12917 tablet_copy_source_session.cc:205] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.697489 18592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe046c56dde144f9adbbbf779067476e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.699074 18592 tablet_copy_client.cc:783] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.699213 18592 tablet_copy_client.cc:647] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.700080 18596 tablet_copy_client.cc:1002] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.761068 18597 ts_tablet_manager.cc:902] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393)
I20240501 14:09:54.761497 18597 tablet_copy_client.cc:308] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:45393
I20240501 14:09:54.761973 12917 tablet_copy_service.cc:140] P a3d67dc22c8a4cc2ba08b836537105e7: Received BeginTabletCopySession request for tablet 4fc05e3adc8e4ed0b45d275725f289f4 from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:40601)
I20240501 14:09:54.762079 12917 tablet_copy_service.cc:161] P a3d67dc22c8a4cc2ba08b836537105e7: Beginning new tablet copy session on tablet 4fc05e3adc8e4ed0b45d275725f289f4 from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:40601: session id = 777595e1371c421fb350f0bfbb7a8035-4fc05e3adc8e4ed0b45d275725f289f4
I20240501 14:09:54.762882 18602 ts_tablet_manager.cc:902] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:54.762920 12917 tablet_copy_source_session.cc:205] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.763411 18602 tablet_copy_client.cc:308] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:54.763832 18597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc05e3adc8e4ed0b45d275725f289f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.763900 13312 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 2a19716d4a9b4acd8a3515b48d9eac32 from peer 18bef23ddc624bd784ea38fc6ef00040 ({username='slave'} at 127.6.153.198:49463)
I20240501 14:09:54.763998 13312 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 2a19716d4a9b4acd8a3515b48d9eac32 from peer 18bef23ddc624bd784ea38fc6ef00040 at {username='slave'} at 127.6.153.198:49463: session id = 18bef23ddc624bd784ea38fc6ef00040-2a19716d4a9b4acd8a3515b48d9eac32
I20240501 14:09:54.764886 13312 tablet_copy_source_session.cc:205] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.764926 18607 ts_tablet_manager.cc:902] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: Initiating tablet copy from peer 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421)
I20240501 14:09:54.765401 18607 tablet_copy_client.cc:308] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:36421
I20240501 14:09:54.765748 18602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a19716d4a9b4acd8a3515b48d9eac32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.765856 13049 tablet_copy_service.cc:140] P 3021737e50984ca4bb2ad151dc4776ff: Received BeginTabletCopySession request for tablet 5053c0f554af4efaa221444af87dd9b9 from peer 3025ea3ec516466fbe5c5c4337733584 ({username='slave'} at 127.6.153.200:42337)
I20240501 14:09:54.765923 18597 tablet_copy_client.cc:783] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.765954 13049 tablet_copy_service.cc:161] P 3021737e50984ca4bb2ad151dc4776ff: Beginning new tablet copy session on tablet 5053c0f554af4efaa221444af87dd9b9 from peer 3025ea3ec516466fbe5c5c4337733584 at {username='slave'} at 127.6.153.200:42337: session id = 3025ea3ec516466fbe5c5c4337733584-5053c0f554af4efaa221444af87dd9b9
I20240501 14:09:54.766059 18597 tablet_copy_client.cc:647] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.766788 13049 tablet_copy_source_session.cc:205] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:54.767782 18602 tablet_copy_client.cc:783] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.767792 18607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5053c0f554af4efaa221444af87dd9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:54.767850 18601 tablet_copy_client.cc:1002] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.767910 18602 tablet_copy_client.cc:647] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.769382 18604 tablet_copy_client.cc:1002] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:54.769731 18607 tablet_copy_client.cc:783] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:54.769913 18607 tablet_copy_client.cc:647] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:54.770753 18609 tablet_copy_client.cc:1002] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:55.934742 18460 tablet_copy_client.cc:523] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:55.936269 18460 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:55.957901 18460 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:55.958189 18460 tablet_bootstrap.cc:492] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:55.958330 18460 ts_tablet_manager.cc:1367] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.000s
I20240501 14:09:55.958663 18460 raft_consensus.cc:348] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:55.958765 18460 raft_consensus.cc:729] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:55.958839 18460 consensus_queue.cc:260] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:55.959117 18460 ts_tablet_manager.cc:1398] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:09:55.959582 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-9d696c814eaa4c718e49dc32df0d6100 received from {username='slave'} at 127.6.153.198:59809
I20240501 14:09:55.959646 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-9d696c814eaa4c718e49dc32df0d6100 on tablet 9d696c814eaa4c718e49dc32df0d6100 with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:55.968956 18465 tablet_copy_client.cc:523] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:55.970130 18465 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:55.978261 14796 raft_consensus.cc:1203] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:55.987681 18470 tablet_copy_client.cc:523] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:55.989116 18470 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:55.989742 18465 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:55.990002 18465 tablet_bootstrap.cc:492] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:55.990157 18465 ts_tablet_manager.cc:1367] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20240501 14:09:55.990567 18465 raft_consensus.cc:348] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:55.990687 18465 raft_consensus.cc:729] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:55.990777 18465 consensus_queue.cc:260] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:55.991036 18465 ts_tablet_manager.cc:1398] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:55.991549 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-7e34fd488c214188aac9990af9de0db4 received from {username='slave'} at 127.6.153.199:52497
I20240501 14:09:55.991634 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-7e34fd488c214188aac9990af9de0db4 on tablet 7e34fd488c214188aac9990af9de0db4 with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:56.007957 18470 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.008183 18470 tablet_bootstrap.cc:492] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:56.008330 18470 ts_tablet_manager.cc:1367] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20240501 14:09:56.008667 18470 raft_consensus.cc:348] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:56.008761 18470 raft_consensus.cc:729] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:56.008846 18470 consensus_queue.cc:260] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:56.009150 18470 ts_tablet_manager.cc:1398] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:56.009842 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-34f677061e42436ab08123227f2be7bf received from {username='slave'} at 127.6.153.200:55411
I20240501 14:09:56.009932 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-34f677061e42436ab08123227f2be7bf on tablet 34f677061e42436ab08123227f2be7bf with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:56.010648 18485 tablet_copy_client.cc:523] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.011809 15060 raft_consensus.cc:1203] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.7->[2.8-2.8] Dedup: 2.8->[]
I20240501 14:09:56.012688 18485 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:56.015079 18501 tablet_copy_client.cc:523] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.016106 18501 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:56.018407 18501 log.cc:824] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: Log is configured to *not* fsync() on all Append() calls
I20240501 14:09:56.018723 18512 tablet_copy_client.cc:523] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.019816 18512 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap starting.
I20240501 14:09:56.022853 18522 tablet_copy_client.cc:523] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.023895 18522 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:56.047775 18485 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.047838 18512 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.048153 18512 tablet_bootstrap.cc:492] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: Bootstrap complete.
I20240501 14:09:56.048267 18485 tablet_bootstrap.cc:492] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:56.048324 18512 ts_tablet_manager.cc:1367] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.002s
I20240501 14:09:56.048605 18485 ts_tablet_manager.cc:1367] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.036s user 0.030s sys 0.006s
I20240501 14:09:56.048880 18552 tablet_copy_client.cc:523] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.049612 18522 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.049854 18522 tablet_bootstrap.cc:492] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:56.049986 18522 ts_tablet_manager.cc:1367] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.026s user 0.019s sys 0.002s
I20240501 14:09:56.050215 18552 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:56.050413 18522 raft_consensus.cc:348] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:56.050535 18522 raft_consensus.cc:729] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:56.050622 18522 consensus_queue.cc:260] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:56.050952 18522 ts_tablet_manager.cc:1398] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:56.050984 18485 raft_consensus.cc:348] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.051223 18485 raft_consensus.cc:729] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:56.051587 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-37047d45ffe14570b28e459afdb66d7e received from {username='slave'} at 127.6.153.198:58259
I20240501 14:09:56.051575 18485 consensus_queue.cc:260] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.051659 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-37047d45ffe14570b28e459afdb66d7e on tablet 37047d45ffe14570b28e459afdb66d7e with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:56.052074 18512 raft_consensus.cc:348] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:56.052525 18512 raft_consensus.cc:729] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Initialized, Role: LEARNER
I20240501 14:09:56.052649 18512 consensus_queue.cc:260] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: true } }
I20240501 14:09:56.052980 18512 ts_tablet_manager.cc:1398] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:09:56.053493 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-2f58cd9e96244a2da7954bdaf9272b82 received from {username='slave'} at 127.6.153.197:57299
I20240501 14:09:56.053566 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 2986bfcc5da34cbca2cad1643b6e35c9-2f58cd9e96244a2da7954bdaf9272b82 on tablet 2f58cd9e96244a2da7954bdaf9272b82 with peer 2986bfcc5da34cbca2cad1643b6e35c9
I20240501 14:09:56.054687 18485 ts_tablet_manager.cc:1398] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240501 14:09:56.055712 18501 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.055968 18501 tablet_bootstrap.cc:492] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:56.055972 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-3ca90427253c4320b2fea78e3fc89d92 received from {username='slave'} at 127.6.153.201:40601
I20240501 14:09:56.056030 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-3ca90427253c4320b2fea78e3fc89d92 on tablet 3ca90427253c4320b2fea78e3fc89d92 with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:56.056118 18501 ts_tablet_manager.cc:1367] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.040s user 0.026s sys 0.000s
I20240501 14:09:56.056556 18501 raft_consensus.cc:348] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.056631 18501 raft_consensus.cc:729] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:56.056710 18501 consensus_queue.cc:260] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.056957 18501 ts_tablet_manager.cc:1398] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:56.057693 13312 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-2ab8fdef27b64f87b89f6e8e4166e0c2 received from {username='slave'} at 127.6.153.201:45235
I20240501 14:09:56.057772 13312 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-2ab8fdef27b64f87b89f6e8e4166e0c2 on tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:56.072253 18552 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.072458 18552 tablet_bootstrap.cc:492] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:56.072576 18552 ts_tablet_manager.cc:1367] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.000s
I20240501 14:09:56.072897 18552 raft_consensus.cc:348] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:56.072990 18552 raft_consensus.cc:729] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:56.073069 18552 consensus_queue.cc:260] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:56.073288 18552 ts_tablet_manager.cc:1398] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:56.073724 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-627fd32202d247c2af434ab8be152102 received from {username='slave'} at 127.6.153.200:55411
I20240501 14:09:56.073794 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-627fd32202d247c2af434ab8be152102 on tablet 627fd32202d247c2af434ab8be152102 with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:56.109833 18577 tablet_copy_client.cc:523] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.111016 18577 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:56.121111 18582 tablet_copy_client.cc:523] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.122356 18582 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap starting.
I20240501 14:09:56.132315 18577 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.132556 18577 tablet_bootstrap.cc:492] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:56.132707 18577 ts_tablet_manager.cc:1367] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.022s user 0.018s sys 0.003s
I20240501 14:09:56.133111 18577 raft_consensus.cc:348] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:56.133234 18577 raft_consensus.cc:729] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:56.133330 18577 consensus_queue.cc:260] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:56.133592 18577 ts_tablet_manager.cc:1398] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:56.134161 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-920be22f0370418885ce155d9874607a received from {username='slave'} at 127.6.153.199:52399
I20240501 14:09:56.134249 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-920be22f0370418885ce155d9874607a on tablet 920be22f0370418885ce155d9874607a with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:56.143934 18582 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.144132 18582 tablet_bootstrap.cc:492] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: Bootstrap complete.
I20240501 14:09:56.144250 18582 ts_tablet_manager.cc:1367] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.002s
I20240501 14:09:56.144560 18582 raft_consensus.cc:348] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:56.144659 18582 raft_consensus.cc:729] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Initialized, Role: LEARNER
I20240501 14:09:56.144737 18582 consensus_queue.cc:260] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: NON_VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: true } }
I20240501 14:09:56.144953 18582 ts_tablet_manager.cc:1398] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:56.145437 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 70c63e471e5b482b8b257c3e7543feb5-73407db0172f454baf2bb44a85add182 received from {username='slave'} at 127.6.153.199:52399
I20240501 14:09:56.145515 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 70c63e471e5b482b8b257c3e7543feb5-73407db0172f454baf2bb44a85add182 on tablet 73407db0172f454baf2bb44a85add182 with peer 70c63e471e5b482b8b257c3e7543feb5
I20240501 14:09:56.154536 18587 tablet_copy_client.cc:523] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.155689 18587 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:56.176024 18587 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.176254 18587 tablet_bootstrap.cc:492] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:56.176393 18587 ts_tablet_manager.cc:1367] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.001s
I20240501 14:09:56.176725 18587 raft_consensus.cc:348] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.176821 18587 raft_consensus.cc:729] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:56.176914 18587 consensus_queue.cc:260] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.177165 18587 ts_tablet_manager.cc:1398] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:56.177624 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-443ae0978f2145658d910e2ad9fa38d0 received from {username='slave'} at 127.6.153.201:36143
I20240501 14:09:56.177702 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-443ae0978f2145658d910e2ad9fa38d0 on tablet 443ae0978f2145658d910e2ad9fa38d0 with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:56.200343 18592 tablet_copy_client.cc:523] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.201597 18592 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:56.222074 18592 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.222322 18592 tablet_bootstrap.cc:492] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:56.222529 18592 ts_tablet_manager.cc:1367] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.021s user 0.020s sys 0.001s
I20240501 14:09:56.222944 18592 raft_consensus.cc:348] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.223081 18592 raft_consensus.cc:729] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:56.223168 18592 consensus_queue.cc:260] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.223448 18592 ts_tablet_manager.cc:1398] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:56.223858 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-fe046c56dde144f9adbbbf779067476e received from {username='slave'} at 127.6.153.201:40601
I20240501 14:09:56.223939 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-fe046c56dde144f9adbbbf779067476e on tablet fe046c56dde144f9adbbbf779067476e with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:56.268086 18597 tablet_copy_client.cc:523] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.269385 18597 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:56.269629 18602 tablet_copy_client.cc:523] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.270960 18602 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap starting.
I20240501 14:09:56.270998 18607 tablet_copy_client.cc:523] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:56.272226 18607 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap starting.
I20240501 14:09:56.292376 18607 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.292717 18607 tablet_bootstrap.cc:492] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: Bootstrap complete.
I20240501 14:09:56.292868 18607 ts_tablet_manager.cc:1367] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: Time spent bootstrapping tablet: real 0.021s user 0.020s sys 0.000s
I20240501 14:09:56.293231 18607 raft_consensus.cc:348] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:56.293349 18607 raft_consensus.cc:729] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Initialized, Role: LEARNER
I20240501 14:09:56.293470 18607 consensus_queue.cc:260] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: true } }
I20240501 14:09:56.293716 18607 ts_tablet_manager.cc:1398] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:56.294183 13049 tablet_copy_service.cc:342] P 3021737e50984ca4bb2ad151dc4776ff: Request end of tablet copy session 3025ea3ec516466fbe5c5c4337733584-5053c0f554af4efaa221444af87dd9b9 received from {username='slave'} at 127.6.153.200:42337
I20240501 14:09:56.294260 13049 tablet_copy_service.cc:433] P 3021737e50984ca4bb2ad151dc4776ff: ending tablet copy session 3025ea3ec516466fbe5c5c4337733584-5053c0f554af4efaa221444af87dd9b9 on tablet 5053c0f554af4efaa221444af87dd9b9 with peer 3025ea3ec516466fbe5c5c4337733584
I20240501 14:09:56.297401 18602 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.297664 18602 tablet_bootstrap.cc:492] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: Bootstrap complete.
I20240501 14:09:56.297824 18602 ts_tablet_manager.cc:1367] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: Time spent bootstrapping tablet: real 0.027s user 0.022s sys 0.003s
I20240501 14:09:56.298233 18602 raft_consensus.cc:348] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:56.298354 18602 raft_consensus.cc:729] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Initialized, Role: LEARNER
I20240501 14:09:56.298460 18602 consensus_queue.cc:260] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: NON_VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: true } }
I20240501 14:09:56.298707 18597 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:56.298740 18602 ts_tablet_manager.cc:1398] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:56.298947 18597 tablet_bootstrap.cc:492] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:56.299096 18597 ts_tablet_manager.cc:1367] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.030s user 0.025s sys 0.000s
I20240501 14:09:56.299234 13312 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 18bef23ddc624bd784ea38fc6ef00040-2a19716d4a9b4acd8a3515b48d9eac32 received from {username='slave'} at 127.6.153.198:49463
I20240501 14:09:56.299324 13312 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 18bef23ddc624bd784ea38fc6ef00040-2a19716d4a9b4acd8a3515b48d9eac32 on tablet 2a19716d4a9b4acd8a3515b48d9eac32 with peer 18bef23ddc624bd784ea38fc6ef00040
I20240501 14:09:56.299485 18597 raft_consensus.cc:348] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.299634 18597 raft_consensus.cc:729] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:56.299865 18597 consensus_queue.cc:260] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:56.300499 18597 ts_tablet_manager.cc:1398] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240501 14:09:56.301012 12917 tablet_copy_service.cc:342] P a3d67dc22c8a4cc2ba08b836537105e7: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-4fc05e3adc8e4ed0b45d275725f289f4 received from {username='slave'} at 127.6.153.201:40601
I20240501 14:09:56.301100 12917 tablet_copy_service.cc:433] P a3d67dc22c8a4cc2ba08b836537105e7: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-4fc05e3adc8e4ed0b45d275725f289f4 on tablet 4fc05e3adc8e4ed0b45d275725f289f4 with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:56.348253 15060 raft_consensus.cc:1203] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.375860 16003 raft_consensus.cc:1203] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.411834 14927 raft_consensus.cc:1203] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.414462 16003 raft_consensus.cc:1203] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.437661 14665 raft_consensus.cc:1203] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.461383 15060 raft_consensus.cc:1203] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.507678 14927 raft_consensus.cc:1203] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.523200 16003 raft_consensus.cc:1203] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.535437 18688 raft_consensus.cc:1050] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:56.535858 18688 consensus_queue.cc:237] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:56.537150 14665 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.537160 13292 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.537160 14796 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.537525 19163 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.537712 18688 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.538072 19017 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.540272 18688 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:56.540448 16003 raft_consensus.cc:1203] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.540606 13292 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:56.542922 14665 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:56.542927 14796 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:56.543054 12747 catalog_manager.cc:5484] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:56.549748 13161 consensus_queue.cc:237] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:56.550653 14796 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.550693 14665 raft_consensus.cc:1261] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.550987 19165 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.551151 19017 consensus_queue.cc:1035] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.552593 19163 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:56.554119 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9d696c814eaa4c718e49dc32df0d6100 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:56.554404 14665 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:56.554697 14796 raft_consensus.cc:2932] T 9d696c814eaa4c718e49dc32df0d6100 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:56.554898 12749 catalog_manager.cc:5484] T 9d696c814eaa4c718e49dc32df0d6100 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:56.562208 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d696c814eaa4c718e49dc32df0d6100 with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:56.562537 19173 tablet_replica.cc:330] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:56.562670 19173 raft_consensus.cc:2229] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:56.562852 19173 raft_consensus.cc:2258] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:56.563503 19173 ts_tablet_manager.cc:1875] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:56.564090 16003 raft_consensus.cc:1203] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.565721 19173 ts_tablet_manager.cc:1888] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:56.565779 19173 log.cc:1196] T 9d696c814eaa4c718e49dc32df0d6100 P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/9d696c814eaa4c718e49dc32df0d6100
I20240501 14:09:56.566212 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 9d696c814eaa4c718e49dc32df0d6100 (table rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]) successfully deleted
I20240501 14:09:56.605945 14927 raft_consensus.cc:1203] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.609786 18094 raft_consensus.cc:1050] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:56.610188 18094 consensus_queue.cc:237] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:56.611554 13029 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:56.611855 15060 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 [term 2 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:56.611902 19176 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.612417 14927 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:56.612399 19176 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.613096 19176 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.614161 18293 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.614529 15060 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.615809 13029 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.617929 12751 catalog_manager.cc:5484] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: config changed from index 8 to 9, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.619015 14927 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.622757 12895 consensus_queue.cc:237] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:56.623656 14927 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:56.623696 15060 raft_consensus.cc:1261] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:56.623975 19176 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:56.624137 19174 consensus_queue.cc:1035] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:56.625468 18094 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.625849 15060 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.625856 14927 raft_consensus.cc:2932] T 34f677061e42436ab08123227f2be7bf P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.626878 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 34f677061e42436ab08123227f2be7bf with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:56.627698 12746 catalog_manager.cc:5484] T 34f677061e42436ab08123227f2be7bf P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 9 to 10, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 2 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.629659 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 34f677061e42436ab08123227f2be7bf with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:56.630192 19180 tablet_replica.cc:330] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:56.630326 19180 raft_consensus.cc:2229] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:56.630511 19180 raft_consensus.cc:2258] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:56.631130 19180 ts_tablet_manager.cc:1875] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:56.631707 14796 raft_consensus.cc:1203] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.633462 19180 ts_tablet_manager.cc:1888] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 14:09:56.633559 19180 log.cc:1196] T 34f677061e42436ab08123227f2be7bf P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/34f677061e42436ab08123227f2be7bf
I20240501 14:09:56.634085 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 34f677061e42436ab08123227f2be7bf (table rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]) successfully deleted
I20240501 14:09:56.806020 14796 raft_consensus.cc:1203] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:56.828328 19176 raft_consensus.cc:1050] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 2986bfcc5da34cbca2cad1643b6e35c9 to VOTER
I20240501 14:09:56.828768 19176 consensus_queue.cc:237] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:56.829934 14665 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.830037 14927 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.830056 13161 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.830410 18094 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.830566 19176 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.830920 19174 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.832573 18094 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:56.834249 13161 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:56.834295 14927 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:56.833760 14665 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:56.835923 12751 catalog_manager.cc:5484] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 7 to 8, 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.851234 12895 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49966
I20240501 14:09:56.851387 12895 raft_consensus.cc:595] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:56.867255 18293 raft_consensus.cc:1050] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:09:56.867606 18293 consensus_queue.cc:237] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:56.868849 13292 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.868956 14927 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.869029 16003 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.869194 19176 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.869370 18094 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.869545 18293 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.871178 18094 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.871599 14927 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.872622 16003 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.873246 13292 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.875023 12746 catalog_manager.cc:5484] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.877712 12895 consensus_queue.cc:237] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:56.878728 16003 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.878778 14927 raft_consensus.cc:1261] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.879043 18094 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.879196 19174 consensus_queue.cc:1035] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.880503 18094 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.881253 16003 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.881968 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3ca90427253c4320b2fea78e3fc89d92 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:56.882081 14927 raft_consensus.cc:2932] T 3ca90427253c4320b2fea78e3fc89d92 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.883404 12749 catalog_manager.cc:5484] T 3ca90427253c4320b2fea78e3fc89d92 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.885680 19291 raft_consensus.cc:1050] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:56.885864 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ca90427253c4320b2fea78e3fc89d92 with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:56.886066 19173 tablet_replica.cc:330] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:56.886092 19291 consensus_queue.cc:237] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:56.886189 19173 raft_consensus.cc:2229] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:56.886354 19173 raft_consensus.cc:2258] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:56.887233 19173 ts_tablet_manager.cc:1875] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:56.887640 15060 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.888001 19336 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.888581 12895 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.889094 19173 ts_tablet_manager.cc:1888] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:56.889153 19173 log.cc:1196] T 3ca90427253c4320b2fea78e3fc89d92 P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/3ca90427253c4320b2fea78e3fc89d92
I20240501 14:09:56.889395 19336 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.889557 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 3ca90427253c4320b2fea78e3fc89d92 (table rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]) successfully deleted
I20240501 14:09:56.889737 14927 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.890450 19338 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.890857 19336 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.891981 12895 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.893316 12751 catalog_manager.cc:5484] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.893385 15060 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.893396 14927 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.896947 13029 consensus_queue.cc:237] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:56.897856 15060 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.897886 14927 raft_consensus.cc:1261] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.898198 19336 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.898653 19336 consensus_queue.cc:1035] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.899581 19338 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.900414 14927 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.900980 15060 raft_consensus.cc:2932] T 5053c0f554af4efaa221444af87dd9b9 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.901085 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5053c0f554af4efaa221444af87dd9b9 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:56.901804 12749 catalog_manager.cc:5484] T 5053c0f554af4efaa221444af87dd9b9 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 8 to 9, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.903846 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 5053c0f554af4efaa221444af87dd9b9 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:56.907590 19342 tablet_replica.cc:330] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:56.907708 19342 raft_consensus.cc:2229] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:56.907869 19342 raft_consensus.cc:2258] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:56.908372 19342 ts_tablet_manager.cc:1875] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:56.910071 18293 raft_consensus.cc:1050] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 3025ea3ec516466fbe5c5c4337733584 to VOTER
I20240501 14:09:56.910387 19342 ts_tablet_manager.cc:1888] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:56.910434 19342 log.cc:1196] T 5053c0f554af4efaa221444af87dd9b9 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/5053c0f554af4efaa221444af87dd9b9
I20240501 14:09:56.910409 18293 consensus_queue.cc:237] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:56.911201 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 5053c0f554af4efaa221444af87dd9b9 (table rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]) successfully deleted
I20240501 14:09:56.911832 14796 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.912011 15060 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.912184 13161 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.912446 19176 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.912756 19176 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.912884 19174 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.914080 19087 raft_consensus.cc:1050] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:09:56.914430 19087 consensus_queue.cc:237] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:56.915875 16003 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.916353 18211 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.916916 14665 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.916985 13161 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.917338 18211 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.917632 18211 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.919798 18211 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.919896 19174 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.920172 13161 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.920205 14796 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.921936 15060 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.922329 12751 catalog_manager.cc:5484] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.922478 16003 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.924244 12746 catalog_manager.cc:5484] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.925195 13161 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:56.926229 14665 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.929559 13292 consensus_queue.cc:237] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:56.930454 16003 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.930727 14665 raft_consensus.cc:1261] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.930760 19344 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.931026 19348 consensus_queue.cc:1035] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.932368 19087 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.932765 16003 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.933678 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:56.934283 14665 raft_consensus.cc:2932] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.934676 12749 catalog_manager.cc:5484] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.936780 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:56.937104 19353 tablet_replica.cc:330] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:56.937214 19353 raft_consensus.cc:2229] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:56.937358 19353 raft_consensus.cc:2258] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:56.937835 19353 ts_tablet_manager.cc:1875] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:56.939432 19353 ts_tablet_manager.cc:1888] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:56.939491 19353 log.cc:1196] T 2ab8fdef27b64f87b89f6e8e4166e0c2 P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/2ab8fdef27b64f87b89f6e8e4166e0c2
I20240501 14:09:56.939867 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 2ab8fdef27b64f87b89f6e8e4166e0c2 (table rebalance_test_table_0 [id=93e679c4938c4441a7d123aaeefe7a5c]) successfully deleted
I20240501 14:09:56.965626 19176 raft_consensus.cc:1050] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:09:56.965975 19176 consensus_queue.cc:237] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:56.967157 16003 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.967157 14665 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.967234 13029 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:56.967567 19174 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.967723 18094 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.967936 19176 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:56.969465 19174 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.969893 14665 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.970129 13029 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.971144 16003 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.972162 12747 catalog_manager.cc:5484] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.974627 12895 consensus_queue.cc:237] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:56.975597 16003 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.975611 14665 raft_consensus.cc:1261] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:56.975910 18293 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.976068 18094 consensus_queue.cc:1035] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:56.977427 19176 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.977886 14665 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.978749 16003 raft_consensus.cc:2932] T fe046c56dde144f9adbbbf779067476e P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:56.978946 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fe046c56dde144f9adbbbf779067476e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:56.979501 12746 catalog_manager.cc:5484] T fe046c56dde144f9adbbbf779067476e P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:56.981652 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet fe046c56dde144f9adbbbf779067476e with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:56.981868 19180 tablet_replica.cc:330] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:56.982007 19180 raft_consensus.cc:2229] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:56.982122 19180 raft_consensus.cc:2258] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:56.982826 19180 ts_tablet_manager.cc:1875] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:56.984439 19180 ts_tablet_manager.cc:1888] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:56.984498 19180 log.cc:1196] T fe046c56dde144f9adbbbf779067476e P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/fe046c56dde144f9adbbbf779067476e
I20240501 14:09:56.984870 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet fe046c56dde144f9adbbbf779067476e (table rebalance_test_table_2 [id=e6a2dd705e2841c18575c3af0dbaa72c]) successfully deleted
I20240501 14:09:57.005581 19163 raft_consensus.cc:1050] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:09:57.006011 19163 consensus_queue.cc:237] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:57.007468 14665 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.007510 16003 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.007550 13292 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.008111 19165 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.008440 19165 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.008738 19165 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.010672 19017 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.011184 13292 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.012400 14665 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.013701 16003 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.014189 12745 catalog_manager.cc:5484] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.016676 19291 raft_consensus.cc:1050] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:57.017091 19291 consensus_queue.cc:237] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.018589 14927 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.018606 14796 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.018922 19338 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.019218 13161 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.019225 19338 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.019587 19338 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.020809 13161 consensus_queue.cc:237] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:57.021780 16003 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:57.021871 14665 raft_consensus.cc:1261] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:57.022114 19017 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.022256 19338 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.022397 19017 consensus_queue.cc:1035] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.022562 14927 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.024400 19017 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.024497 13160 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.024919 14796 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.025424 12746 catalog_manager.cc:5484] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:57.027109 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 443ae0978f2145658d910e2ad9fa38d0 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:57.027683 14665 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.028194 13029 consensus_queue.cc:237] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.028951 12745 catalog_manager.cc:5484] T 443ae0978f2145658d910e2ad9fa38d0 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:57.029526 16003 raft_consensus.cc:2932] T 443ae0978f2145658d910e2ad9fa38d0 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.030233 14796 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:57.030532 19338 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.031378 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 443ae0978f2145658d910e2ad9fa38d0 with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:57.031692 14927 raft_consensus.cc:1261] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:57.031967 19173 tablet_replica.cc:330] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:57.032105 19173 raft_consensus.cc:2229] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:57.032291 19173 raft_consensus.cc:2258] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:57.032650 19338 consensus_queue.cc:1035] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.032938 19173 ts_tablet_manager.cc:1875] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:57.033526 19291 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.033933 14796 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.034749 19173 ts_tablet_manager.cc:1888] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:57.034803 19173 log.cc:1196] T 443ae0978f2145658d910e2ad9fa38d0 P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/443ae0978f2145658d910e2ad9fa38d0
I20240501 14:09:57.035210 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 443ae0978f2145658d910e2ad9fa38d0 (table rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]) successfully deleted
I20240501 14:09:57.035477 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 920be22f0370418885ce155d9874607a with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:57.035849 14927 raft_consensus.cc:2932] T 920be22f0370418885ce155d9874607a P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.036341 12745 catalog_manager.cc:5484] T 920be22f0370418885ce155d9874607a P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.038839 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 920be22f0370418885ce155d9874607a with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:57.039003 19353 tablet_replica.cc:330] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:57.039113 19353 raft_consensus.cc:2229] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:57.039258 19353 raft_consensus.cc:2258] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:57.039804 19353 ts_tablet_manager.cc:1875] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:57.041319 19353 ts_tablet_manager.cc:1888] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:57.041383 19353 log.cc:1196] T 920be22f0370418885ce155d9874607a P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/920be22f0370418885ce155d9874607a
I20240501 14:09:57.041775 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 920be22f0370418885ce155d9874607a (table rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]) successfully deleted
I20240501 14:09:57.057269 19163 raft_consensus.cc:1050] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:57.057634 19163 consensus_queue.cc:237] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.058846 15060 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.058897 14927 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.058965 13292 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.059260 19017 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.059458 19163 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.059689 19165 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.061483 19017 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.061883 14927 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.061954 15060 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.064399 12747 catalog_manager.cc:5484] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.065013 13292 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.096773 18094 raft_consensus.cc:1050] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:09:57.097182 18094 consensus_queue.cc:237] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:57.097330 12895 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "627fd32202d247c2af434ab8be152102"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50016
I20240501 14:09:57.097410 12895 raft_consensus.cc:595] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.098441 13160 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.098479 14796 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.098806 19176 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.099081 16003 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.099421 19176 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.099946 18094 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.101060 19176 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.101533 16003 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.103291 13160 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.103718 12747 catalog_manager.cc:5484] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:57.105600 14796 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.107590 19174 raft_consensus.cc:1050] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:57.107939 19174 consensus_queue.cc:237] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:57.109335 14797 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.109701 18094 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.109850 14927 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.110199 13292 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.110283 18094 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.111522 18094 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.113632 13292 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.113220 12895 consensus_queue.cc:237] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:57.114522 14797 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.113363 18094 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.115196 16003 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:57.115603 19176 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.116786 12745 catalog_manager.cc:5484] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.117089 14927 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.119406 14797 raft_consensus.cc:1261] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer a3d67dc22c8a4cc2ba08b836537105e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:57.119280 19176 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.119762 18094 consensus_queue.cc:1035] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.120749 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4fc05e3adc8e4ed0b45d275725f289f4 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:57.120951 14797 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.122613 16003 raft_consensus.cc:2932] T 4fc05e3adc8e4ed0b45d275725f289f4 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:57.123122 12745 catalog_manager.cc:5484] T 4fc05e3adc8e4ed0b45d275725f289f4 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:57.125566 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fc05e3adc8e4ed0b45d275725f289f4 with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:57.125769 19353 tablet_replica.cc:330] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:57.125913 19353 raft_consensus.cc:2229] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:57.126088 19353 raft_consensus.cc:2258] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:57.126717 19353 ts_tablet_manager.cc:1875] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:57.128789 19353 ts_tablet_manager.cc:1888] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:57.128855 19353 log.cc:1196] T 4fc05e3adc8e4ed0b45d275725f289f4 P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/4fc05e3adc8e4ed0b45d275725f289f4
I20240501 14:09:57.129283 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 4fc05e3adc8e4ed0b45d275725f289f4 (table rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]) successfully deleted
I20240501 14:09:57.132115 12895 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50032
I20240501 14:09:57.132205 12895 raft_consensus.cc:595] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.145213 13160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4"
dest_uuid: "b6ea309b0a4a44dc9415766897152739"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38828
I20240501 14:09:57.145315 13160 raft_consensus.cc:595] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.153503 12895 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50050
I20240501 14:09:57.153604 12895 raft_consensus.cc:595] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.165167 19336 raft_consensus.cc:1050] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: attempting to promote NON_VOTER 70c63e471e5b482b8b257c3e7543feb5 to VOTER
I20240501 14:09:57.165181 13160 consensus_queue.cc:237] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:57.165565 19336 consensus_queue.cc:237] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.166811 13290 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:57.167129 12895 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.167186 18688 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.167238 15060 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:57.167379 13096 consensus_peers.cc:485] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ac389d29e274b15b5f3db598a04f5e7. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:57.167551 14797 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.167848 18328 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.168164 18328 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.168444 14927 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 LEARNER]: Refusing update from remote peer 3021737e50984ca4bb2ad151dc4776ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.168852 19165 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.169123 19163 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.169230 13292 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.170156 19291 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.171566 12747 catalog_manager.cc:5484] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.171893 19338 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.172116 15060 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.172294 14797 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.174052 12895 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.174489 12749 catalog_manager.cc:5484] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3021737e50984ca4bb2ad151dc4776ff" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.183089 13160 consensus_queue.cc:237] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:57.184485 13029 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:57.184767 14665 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:57.184798 13096 consensus_peers.cc:485] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f63509c2039d49f09eee28a45e03ad81. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:57.184947 18688 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.185142 19017 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.186621 18688 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.187009 13029 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.187453 14665 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.189636 12745 catalog_manager.cc:5484] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.196879 14927 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:57.201018 13292 consensus_queue.cc:237] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:57.202554 13029 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:57.203092 14665 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:57.203474 19087 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.203665 19348 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240501 14:09:57.204118 13227 consensus_peers.cc:485] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15661c749d7a4cd688d4d666a5eca1c8. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:57.207978 19348 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.208477 13029 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.211138 12747 catalog_manager.cc:5484] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.211499 14665 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.221454 13160 consensus_queue.cc:237] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:57.223006 15060 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:57.223080 13029 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:57.223414 13096 consensus_peers.cc:485] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bdfa21a72f85470396cc525fba5e379d. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:57.223570 19163 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.223729 19165 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.225430 18688 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.225795 15060 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.227205 13029 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.229259 12749 catalog_manager.cc:5484] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.263914 12895 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "627fd32202d247c2af434ab8be152102"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50074
I20240501 14:09:57.264030 12895 raft_consensus.cc:595] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.272311 13029 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "73407db0172f454baf2bb44a85add182"
dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47908
I20240501 14:09:57.272449 13029 raft_consensus.cc:595] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.273880 19174 raft_consensus.cc:979] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: : Instructing follower b6ea309b0a4a44dc9415766897152739 to start an election
I20240501 14:09:57.273999 18094 raft_consensus.cc:1067] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Signalling peer b6ea309b0a4a44dc9415766897152739 to start an election
I20240501 14:09:57.274471 13160 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82"
dest_uuid: "b6ea309b0a4a44dc9415766897152739"
from {username='slave'} at 127.6.153.193:60555
I20240501 14:09:57.274623 13160 raft_consensus.cc:482] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:57.274689 13160 raft_consensus.cc:3037] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.275770 13160 raft_consensus.cc:504] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:57.276250 13160 leader_election.cc:288] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 2 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347), 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197:37879)
I20240501 14:09:57.276453 12895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:57.276551 12895 raft_consensus.cc:3032] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:57.276583 12895 raft_consensus.cc:729] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:57.276723 12895 consensus_queue.cc:260] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:57.276880 12895 raft_consensus.cc:3037] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.276863 14927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "70c63e471e5b482b8b257c3e7543feb5"
I20240501 14:09:57.276868 14665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f58cd9e96244a2da7954bdaf9272b82" candidate_uuid: "b6ea309b0a4a44dc9415766897152739" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "2986bfcc5da34cbca2cad1643b6e35c9"
I20240501 14:09:57.276983 14927 raft_consensus.cc:3037] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.277034 14665 raft_consensus.cc:3037] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.277823 12895 raft_consensus.cc:2454] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 2.
I20240501 14:09:57.278035 14665 raft_consensus.cc:2454] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 2.
I20240501 14:09:57.278364 13095 leader_election.cc:302] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2986bfcc5da34cbca2cad1643b6e35c9, a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:57.278532 19163 raft_consensus.cc:2781] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:57.278606 19163 raft_consensus.cc:686] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 2 LEADER]: Becoming Leader. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:57.278604 14927 raft_consensus.cc:2454] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ea309b0a4a44dc9415766897152739 in term 2.
I20240501 14:09:57.278743 19163 consensus_queue.cc:237] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:57.280427 12746 catalog_manager.cc:5484] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 reported cstate change: term changed from 1 to 2, leader changed from a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) to b6ea309b0a4a44dc9415766897152739 (127.6.153.195). New cstate: current_term: 2 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.284430 12895 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50098
I20240501 14:09:57.284528 12895 raft_consensus.cc:595] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.297576 19174 raft_consensus.cc:979] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: : Instructing follower 18bef23ddc624bd784ea38fc6ef00040 to start an election
I20240501 14:09:57.297677 19174 raft_consensus.cc:1067] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Signalling peer 18bef23ddc624bd784ea38fc6ef00040 to start an election
I20240501 14:09:57.297644 13160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4"
dest_uuid: "b6ea309b0a4a44dc9415766897152739"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38890
I20240501 14:09:57.297753 13160 raft_consensus.cc:595] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.298125 14797 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "627fd32202d247c2af434ab8be152102"
dest_uuid: "18bef23ddc624bd784ea38fc6ef00040"
from {username='slave'} at 127.6.153.193:46137
I20240501 14:09:57.298307 14797 raft_consensus.cc:482] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:57.298413 14797 raft_consensus.cc:3037] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.299180 14797 raft_consensus.cc:504] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:57.300411 14797 leader_election.cc:288] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [CANDIDATE]: Term 2 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200:34743)
I20240501 14:09:57.301044 12895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fd32202d247c2af434ab8be152102" candidate_uuid: "18bef23ddc624bd784ea38fc6ef00040" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:57.301149 12895 raft_consensus.cc:3032] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:57.301193 12895 raft_consensus.cc:729] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:57.301188 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fd32202d247c2af434ab8be152102" candidate_uuid: "18bef23ddc624bd784ea38fc6ef00040" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:57.301306 13160 raft_consensus.cc:3037] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.301327 12895 consensus_queue.cc:260] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:57.301524 12895 raft_consensus.cc:3037] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.302233 13160 raft_consensus.cc:2454] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18bef23ddc624bd784ea38fc6ef00040 in term 2.
I20240501 14:09:57.302426 12895 raft_consensus.cc:2454] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18bef23ddc624bd784ea38fc6ef00040 in term 2.
I20240501 14:09:57.302686 14729 leader_election.cc:302] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 18bef23ddc624bd784ea38fc6ef00040, a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:57.302904 18993 raft_consensus.cc:2781] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:57.303076 18993 raft_consensus.cc:686] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 2 LEADER]: Becoming Leader. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Running, Role: LEADER
I20240501 14:09:57.303385 18993 consensus_queue.cc:237] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:57.304778 15060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fd32202d247c2af434ab8be152102" candidate_uuid: "18bef23ddc624bd784ea38fc6ef00040" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "3025ea3ec516466fbe5c5c4337733584"
I20240501 14:09:57.304949 15060 raft_consensus.cc:3037] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.305984 15060 raft_consensus.cc:2454] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18bef23ddc624bd784ea38fc6ef00040 in term 2.
I20240501 14:09:57.306653 12746 catalog_manager.cc:5484] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: term changed from 1 to 2, leader changed from a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) to 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198). New cstate: current_term: 2 leader_uuid: "18bef23ddc624bd784ea38fc6ef00040" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.315214 13160 consensus_queue.cc:237] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:57.316519 12895 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:09:57.316627 15060 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 14:09:57.316838 13096 consensus_peers.cc:485] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 -> Peer 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197): Couldn't send request to peer 777595e1371c421fb350f0bfbb7a8035. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2affe37c7174ff1be13ba4807d1fffe. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:09:57.316936 19165 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.317092 19017 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:09:57.318689 19163 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.319171 12895 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.319729 15060 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } } }
I20240501 14:09:57.321357 12746 catalog_manager.cc:5484] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 6 to 7, NON_VOTER 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.364120 13029 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "73407db0172f454baf2bb44a85add182"
dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47926
I20240501 14:09:57.364243 13029 raft_consensus.cc:595] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.374576 12895 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50112
I20240501 14:09:57.374693 12895 raft_consensus.cc:595] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.390293 13160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4"
dest_uuid: "b6ea309b0a4a44dc9415766897152739"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38958
I20240501 14:09:57.390398 13160 raft_consensus.cc:595] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.403848 19344 raft_consensus.cc:1050] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 18bef23ddc624bd784ea38fc6ef00040 to VOTER
I20240501 14:09:57.404215 19344 consensus_queue.cc:237] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:57.405409 13160 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.405467 14927 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.405509 14797 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:57.405822 19344 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.405998 18211 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.406157 19087 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:57.407938 19344 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.408351 14797 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.408336 14927 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.410128 13160 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.410449 12745 catalog_manager.cc:5484] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 7 to 8, 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.413704 13292 consensus_queue.cc:237] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:57.414599 14797 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:57.414655 14927 raft_consensus.cc:1261] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:57.414906 18211 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.415079 19348 consensus_queue.cc:1035] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.416438 18211 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.416812 14797 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.417560 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2a19716d4a9b4acd8a3515b48d9eac32 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:57.418475 12749 catalog_manager.cc:5484] T 2a19716d4a9b4acd8a3515b48d9eac32 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:57.419206 14927 raft_consensus.cc:2932] T 2a19716d4a9b4acd8a3515b48d9eac32 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:57.420490 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a19716d4a9b4acd8a3515b48d9eac32 with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:57.420667 19353 tablet_replica.cc:330] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:57.420789 19353 raft_consensus.cc:2229] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:57.420954 19353 raft_consensus.cc:2258] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:57.421571 19353 ts_tablet_manager.cc:1875] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:57.423197 19353 ts_tablet_manager.cc:1888] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:57.423242 19353 log.cc:1196] T 2a19716d4a9b4acd8a3515b48d9eac32 P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/2a19716d4a9b4acd8a3515b48d9eac32
I20240501 14:09:57.423679 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 2a19716d4a9b4acd8a3515b48d9eac32 (table rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]) successfully deleted
I20240501 14:09:57.513447 19017 raft_consensus.cc:979] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: : Instructing follower 3025ea3ec516466fbe5c5c4337733584 to start an election
I20240501 14:09:57.513542 19017 raft_consensus.cc:1067] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Signalling peer 3025ea3ec516466fbe5c5c4337733584 to start an election
I20240501 14:09:57.513958 15060 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4"
dest_uuid: "3025ea3ec516466fbe5c5c4337733584"
from {username='slave'} at 127.6.153.195:47333
I20240501 14:09:57.514117 15060 raft_consensus.cc:482] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:57.514168 15060 raft_consensus.cc:3037] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.514890 15060 raft_consensus.cc:504] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.515925 15060 leader_election.cc:288] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [CANDIDATE]: Term 2 election: Requested vote from peers b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347)
I20240501 14:09:57.516185 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4" candidate_uuid: "3025ea3ec516466fbe5c5c4337733584" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:09:57.516307 13160 raft_consensus.cc:3032] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:57.516361 13160 raft_consensus.cc:729] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:09:57.516515 13160 consensus_queue.cc:260] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.516708 13160 raft_consensus.cc:3037] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.517627 13160 raft_consensus.cc:2454] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3025ea3ec516466fbe5c5c4337733584 in term 2.
I20240501 14:09:57.520164 13292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4" candidate_uuid: "3025ea3ec516466fbe5c5c4337733584" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:57.520288 13292 raft_consensus.cc:3037] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.520784 14927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e34fd488c214188aac9990af9de0db4" candidate_uuid: "3025ea3ec516466fbe5c5c4337733584" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "70c63e471e5b482b8b257c3e7543feb5"
I20240501 14:09:57.520916 14927 raft_consensus.cc:3037] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.521186 13292 raft_consensus.cc:2454] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3025ea3ec516466fbe5c5c4337733584 in term 2.
I20240501 14:09:57.521473 14996 leader_election.cc:302] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3025ea3ec516466fbe5c5c4337733584, 879199fbf918416cb23b0376dde0aa6e, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:09:57.521705 18996 raft_consensus.cc:2781] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:57.521806 14927 raft_consensus.cc:2454] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3025ea3ec516466fbe5c5c4337733584 in term 2.
I20240501 14:09:57.521899 18996 raft_consensus.cc:686] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 2 LEADER]: Becoming Leader. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Running, Role: LEADER
I20240501 14:09:57.522243 18996 consensus_queue.cc:237] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.524654 12749 catalog_manager.cc:5484] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: term changed from 1 to 2, leader changed from b6ea309b0a4a44dc9415766897152739 (127.6.153.195) to 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200). New cstate: current_term: 2 leader_uuid: "3025ea3ec516466fbe5c5c4337733584" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.598184 18328 raft_consensus.cc:979] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: : Instructing follower 18bef23ddc624bd784ea38fc6ef00040 to start an election
I20240501 14:09:57.598309 19291 raft_consensus.cc:1067] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Signalling peer 18bef23ddc624bd784ea38fc6ef00040 to start an election
I20240501 14:09:57.598917 14797 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "73407db0172f454baf2bb44a85add182"
dest_uuid: "18bef23ddc624bd784ea38fc6ef00040"
from {username='slave'} at 127.6.153.194:58135
I20240501 14:09:57.599046 14797 raft_consensus.cc:482] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:57.599107 14797 raft_consensus.cc:3037] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.599898 14797 raft_consensus.cc:504] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.600462 14797 leader_election.cc:288] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [CANDIDATE]: Term 2 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199:38347)
I20240501 14:09:57.600703 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73407db0172f454baf2bb44a85add182" candidate_uuid: "18bef23ddc624bd784ea38fc6ef00040" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:09:57.600812 13029 raft_consensus.cc:3032] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:57.600755 12895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73407db0172f454baf2bb44a85add182" candidate_uuid: "18bef23ddc624bd784ea38fc6ef00040" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:57.600850 13029 raft_consensus.cc:729] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3021737e50984ca4bb2ad151dc4776ff, State: Running, Role: LEADER
I20240501 14:09:57.600893 12895 raft_consensus.cc:3037] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.600996 13029 consensus_queue.cc:260] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.601169 13029 raft_consensus.cc:3037] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.601756 12895 raft_consensus.cc:2454] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18bef23ddc624bd784ea38fc6ef00040 in term 2.
I20240501 14:09:57.601853 13029 raft_consensus.cc:2454] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18bef23ddc624bd784ea38fc6ef00040 in term 2.
I20240501 14:09:57.602303 14729 leader_election.cc:302] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 18bef23ddc624bd784ea38fc6ef00040, 3021737e50984ca4bb2ad151dc4776ff, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:57.602463 18993 raft_consensus.cc:2781] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:57.602551 18993 raft_consensus.cc:686] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 2 LEADER]: Becoming Leader. State: Replica: 18bef23ddc624bd784ea38fc6ef00040, State: Running, Role: LEADER
I20240501 14:09:57.602694 18993 consensus_queue.cc:237] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:57.604645 12749 catalog_manager.cc:5484] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 reported cstate change: term changed from 1 to 2, leader changed from 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) to 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198). New cstate: current_term: 2 leader_uuid: "18bef23ddc624bd784ea38fc6ef00040" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.605505 14927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73407db0172f454baf2bb44a85add182" candidate_uuid: "18bef23ddc624bd784ea38fc6ef00040" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "70c63e471e5b482b8b257c3e7543feb5"
I20240501 14:09:57.605619 14927 raft_consensus.cc:3037] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.606546 14927 raft_consensus.cc:2454] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18bef23ddc624bd784ea38fc6ef00040 in term 2.
I20240501 14:09:57.644376 12895 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e"
dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50132
I20240501 14:09:57.644493 12895 raft_consensus.cc:595] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:09:57.665606 14665 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 2 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:57.665994 19017 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.667728 14927 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:57.668382 19017 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.671504 19174 raft_consensus.cc:979] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: : Instructing follower 70c63e471e5b482b8b257c3e7543feb5 to start an election
I20240501 14:09:57.671589 19174 raft_consensus.cc:1067] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Signalling peer 70c63e471e5b482b8b257c3e7543feb5 to start an election
I20240501 14:09:57.672297 12895 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:57.672523 14927 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e"
dest_uuid: "70c63e471e5b482b8b257c3e7543feb5"
from {username='slave'} at 127.6.153.193:47101
I20240501 14:09:57.672638 14927 raft_consensus.cc:482] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:09:57.672690 14927 raft_consensus.cc:3037] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.673640 14927 raft_consensus.cc:504] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:57.674109 14927 leader_election.cc:288] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [CANDIDATE]: Term 2 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), 18bef23ddc624bd784ea38fc6ef00040 (127.6.153.198:38509)
I20240501 14:09:57.674602 12895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e" candidate_uuid: "70c63e471e5b482b8b257c3e7543feb5" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:09:57.674708 12895 raft_consensus.cc:3032] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:09:57.674746 12895 raft_consensus.cc:729] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3d67dc22c8a4cc2ba08b836537105e7, State: Running, Role: LEADER
I20240501 14:09:57.674795 19017 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.674835 13292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e" candidate_uuid: "70c63e471e5b482b8b257c3e7543feb5" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:09:57.674875 12895 consensus_queue.cc:260] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:57.674955 13292 raft_consensus.cc:3037] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.675046 12895 raft_consensus.cc:3037] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.675294 13160 consensus_queue.cc:237] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } }
I20240501 14:09:57.675966 13292 raft_consensus.cc:2454] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70c63e471e5b482b8b257c3e7543feb5 in term 2.
I20240501 14:09:57.676023 12895 raft_consensus.cc:2454] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70c63e471e5b482b8b257c3e7543feb5 in term 2.
I20240501 14:09:57.676518 14864 leader_election.cc:302] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 70c63e471e5b482b8b257c3e7543feb5, 879199fbf918416cb23b0376dde0aa6e, a3d67dc22c8a4cc2ba08b836537105e7; no voters:
I20240501 14:09:57.676663 17570 raft_consensus.cc:2781] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:09:57.676820 17570 raft_consensus.cc:686] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 2 LEADER]: Becoming Leader. State: Replica: 70c63e471e5b482b8b257c3e7543feb5, State: Running, Role: LEADER
I20240501 14:09:57.676875 14665 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 2 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:57.676949 17570 consensus_queue.cc:237] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:57.677196 19017 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:57.677978 14927 raft_consensus.cc:1261] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:57.678448 18688 consensus_queue.cc:1035] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:57.679034 18688 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P b6ea309b0a4a44dc9415766897152739 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:57.679425 14665 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:57.681614 14797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37047d45ffe14570b28e459afdb66d7e" candidate_uuid: "70c63e471e5b482b8b257c3e7543feb5" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "18bef23ddc624bd784ea38fc6ef00040"
I20240501 14:09:57.681581 12749 catalog_manager.cc:5484] T 2f58cd9e96244a2da7954bdaf9272b82 P 2986bfcc5da34cbca2cad1643b6e35c9 reported cstate change: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 2 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:57.681790 14797 raft_consensus.cc:3037] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:09:57.682472 14927 raft_consensus.cc:2932] T 2f58cd9e96244a2da7954bdaf9272b82 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } }
I20240501 14:09:57.682744 14797 raft_consensus.cc:2454] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70c63e471e5b482b8b257c3e7543feb5 in term 2.
I20240501 14:09:57.682879 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2f58cd9e96244a2da7954bdaf9272b82 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:57.683519 12746 catalog_manager.cc:5484] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: term changed from 1 to 2, leader changed from a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) to 70c63e471e5b482b8b257c3e7543feb5 (127.6.153.199). New cstate: current_term: 2 leader_uuid: "70c63e471e5b482b8b257c3e7543feb5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:57.684334 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f58cd9e96244a2da7954bdaf9272b82 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:57.684872 19748 tablet_replica.cc:330] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:57.685066 19748 raft_consensus.cc:2229] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:57.685218 19748 raft_consensus.cc:2258] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:57.685779 19748 ts_tablet_manager.cc:1875] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:57.688185 19748 ts_tablet_manager.cc:1888] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 14:09:57.688314 19748 log.cc:1196] T 2f58cd9e96244a2da7954bdaf9272b82 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/2f58cd9e96244a2da7954bdaf9272b82
I20240501 14:09:57.688812 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 2f58cd9e96244a2da7954bdaf9272b82 (table rebalance_test_table_8 [id=4336ca3416d44785b6fa3f8d8c60561f]) successfully deleted
I20240501 14:09:57.748472 19814 ts_tablet_manager.cc:902] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:57.748992 19814 tablet_copy_client.cc:308] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:57.749433 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet bdfa21a72f85470396cc525fba5e379d from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:36143)
I20240501 14:09:57.749532 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet bdfa21a72f85470396cc525fba5e379d from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:36143: session id = 777595e1371c421fb350f0bfbb7a8035-bdfa21a72f85470396cc525fba5e379d
I20240501 14:09:57.750382 13181 tablet_copy_source_session.cc:205] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:57.751319 19814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdfa21a72f85470396cc525fba5e379d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:57.753377 19814 tablet_copy_client.cc:783] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:57.753513 19814 tablet_copy_client.cc:647] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:57.754720 19816 tablet_copy_client.cc:1002] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:57.786437 19835 ts_tablet_manager.cc:902] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:57.786996 19835 tablet_copy_client.cc:308] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:57.787736 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet 6ac389d29e274b15b5f3db598a04f5e7 from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:36143)
I20240501 14:09:57.787835 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet 6ac389d29e274b15b5f3db598a04f5e7 from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:36143: session id = 777595e1371c421fb350f0bfbb7a8035-6ac389d29e274b15b5f3db598a04f5e7
I20240501 14:09:57.788704 13181 tablet_copy_source_session.cc:205] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:57.789430 19835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac389d29e274b15b5f3db598a04f5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:57.790988 19835 tablet_copy_client.cc:783] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:57.791227 19835 tablet_copy_client.cc:647] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:57.792129 19837 tablet_copy_client.cc:1002] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:57.797394 19840 ts_tablet_manager.cc:902] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309)
I20240501 14:09:57.797787 19840 tablet_copy_client.cc:308] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:33309
I20240501 14:09:57.798171 13312 tablet_copy_service.cc:140] P 879199fbf918416cb23b0376dde0aa6e: Received BeginTabletCopySession request for tablet 15661c749d7a4cd688d4d666a5eca1c8 from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:45235)
I20240501 14:09:57.798269 13312 tablet_copy_service.cc:161] P 879199fbf918416cb23b0376dde0aa6e: Beginning new tablet copy session on tablet 15661c749d7a4cd688d4d666a5eca1c8 from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:45235: session id = 777595e1371c421fb350f0bfbb7a8035-15661c749d7a4cd688d4d666a5eca1c8
I20240501 14:09:57.798945 13312 tablet_copy_source_session.cc:205] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:57.799604 19840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15661c749d7a4cd688d4d666a5eca1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:57.801287 19840 tablet_copy_client.cc:783] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:57.801416 19840 tablet_copy_client.cc:647] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:57.801872 19845 ts_tablet_manager.cc:902] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:57.802345 19845 tablet_copy_client.cc:308] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:57.802383 19844 tablet_copy_client.cc:1002] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:57.802719 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet f63509c2039d49f09eee28a45e03ad81 from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:36143)
I20240501 14:09:57.802804 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet f63509c2039d49f09eee28a45e03ad81 from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:36143: session id = 777595e1371c421fb350f0bfbb7a8035-f63509c2039d49f09eee28a45e03ad81
I20240501 14:09:57.803427 13181 tablet_copy_source_session.cc:205] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:57.804037 19845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63509c2039d49f09eee28a45e03ad81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:57.805392 19845 tablet_copy_client.cc:783] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:57.805557 19845 tablet_copy_client.cc:647] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:57.806344 19849 tablet_copy_client.cc:1002] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:57.827054 12895 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:57.827518 19577 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.857858 19850 ts_tablet_manager.cc:902] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: Initiating tablet copy from peer b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667)
I20240501 14:09:57.858242 13160 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:57.858284 19850 tablet_copy_client.cc:308] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:38667
I20240501 14:09:57.858629 19577 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.858732 13181 tablet_copy_service.cc:140] P b6ea309b0a4a44dc9415766897152739: Received BeginTabletCopySession request for tablet c2affe37c7174ff1be13ba4807d1fffe from peer 777595e1371c421fb350f0bfbb7a8035 ({username='slave'} at 127.6.153.201:36143)
I20240501 14:09:57.858815 13181 tablet_copy_service.cc:161] P b6ea309b0a4a44dc9415766897152739: Beginning new tablet copy session on tablet c2affe37c7174ff1be13ba4807d1fffe from peer 777595e1371c421fb350f0bfbb7a8035 at {username='slave'} at 127.6.153.201:36143: session id = 777595e1371c421fb350f0bfbb7a8035-c2affe37c7174ff1be13ba4807d1fffe
I20240501 14:09:57.859650 13181 tablet_copy_source_session.cc:205] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:09:57.860386 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2affe37c7174ff1be13ba4807d1fffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:09:57.861187 15060 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:57.862354 19856 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:57.863260 19850 tablet_copy_client.cc:783] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 0 data blocks...
I20240501 14:09:57.863410 19850 tablet_copy_client.cc:647] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Starting download of 1 WAL segments...
I20240501 14:09:57.864682 19854 tablet_copy_client.cc:1002] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Injecting latency into file download: 1500
I20240501 14:09:57.934556 13292 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:57.934937 18996 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.042835 14927 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:58.043241 18996 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.045802 13160 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:58.046175 19904 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.051726 15059 consensus_queue.cc:237] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:58.052706 13292 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:58.052747 14927 raft_consensus.cc:1261] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:58.053045 19904 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:58.053164 18996 consensus_queue.cc:1035] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:58.054483 18996 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 3025ea3ec516466fbe5c5c4337733584 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:58.054967 14927 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:58.055060 13292 raft_consensus.cc:2932] T 7e34fd488c214188aac9990af9de0db4 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:58.056443 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7e34fd488c214188aac9990af9de0db4 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:58.057284 12746 catalog_manager.cc:5484] T 7e34fd488c214188aac9990af9de0db4 P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 8 to 10, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 2 leader_uuid: "3025ea3ec516466fbe5c5c4337733584" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:58.059271 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e34fd488c214188aac9990af9de0db4 with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:55598
I20240501 14:09:58.059598 19907 tablet_replica.cc:330] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:09:58.059736 19907 raft_consensus.cc:2229] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:58.059846 19907 raft_consensus.cc:2258] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:58.060462 19907 ts_tablet_manager.cc:1875] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:58.062320 19907 ts_tablet_manager.cc:1888] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 14:09:58.062381 19907 log.cc:1196] T 7e34fd488c214188aac9990af9de0db4 P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/7e34fd488c214188aac9990af9de0db4
I20240501 14:09:58.062839 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet 7e34fd488c214188aac9990af9de0db4 (table rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]) successfully deleted
I20240501 14:09:58.081293 13292 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:58.081634 17570 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.091660 13029 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:58.092007 18993 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.101707 14797 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:58.102044 17570 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.103986 12895 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:58.104318 19908 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.105656 14927 consensus_queue.cc:237] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } }
I20240501 14:09:58.106684 13292 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:58.107008 19908 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:58.107211 14797 raft_consensus.cc:1261] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 2 FOLLOWER]: Refusing update from remote peer 70c63e471e5b482b8b257c3e7543feb5: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:58.107650 19908 consensus_queue.cc:1035] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:58.108620 17570 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:58.109458 13290 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:58.110610 12741 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 37047d45ffe14570b28e459afdb66d7e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:58.111469 12746 catalog_manager.cc:5484] T 37047d45ffe14570b28e459afdb66d7e P 70c63e471e5b482b8b257c3e7543feb5 reported cstate change: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 2 leader_uuid: "70c63e471e5b482b8b257c3e7543feb5" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:09:58.111675 14797 raft_consensus.cc:2932] T 37047d45ffe14570b28e459afdb66d7e P 18bef23ddc624bd784ea38fc6ef00040 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } }
I20240501 14:09:58.113855 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 37047d45ffe14570b28e459afdb66d7e with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:58.114042 19748 tablet_replica.cc:330] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:58.114172 19748 raft_consensus.cc:2229] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:58.114279 19748 raft_consensus.cc:2258] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:58.115016 19748 ts_tablet_manager.cc:1875] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:58.116039 12895 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:58.116361 18993 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.116921 19748 ts_tablet_manager.cc:1888] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 14:09:58.116993 19748 log.cc:1196] T 37047d45ffe14570b28e459afdb66d7e P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/37047d45ffe14570b28e459afdb66d7e
I20240501 14:09:58.117511 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 37047d45ffe14570b28e459afdb66d7e (table rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]) successfully deleted
I20240501 14:09:58.118845 14927 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:09:58.119155 19909 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:58.348016 14797 consensus_queue.cc:237] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } }
I20240501 14:09:58.349067 15059 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:58.349198 13160 raft_consensus.cc:1261] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:58.349452 19577 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:58.349659 19909 consensus_queue.cc:1035] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:58.351094 19856 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 18bef23ddc624bd784ea38fc6ef00040 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:58.351486 15059 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:58.351613 13160 raft_consensus.cc:2932] T 627fd32202d247c2af434ab8be152102 P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:58.353550 12747 catalog_manager.cc:5484] T 627fd32202d247c2af434ab8be152102 P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: config changed from index 8 to 10, VOTER a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193) evicted. New cstate: current_term: 2 leader_uuid: "18bef23ddc624bd784ea38fc6ef00040" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } }
I20240501 14:09:58.355605 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 627fd32202d247c2af434ab8be152102 with delete_type TABLET_DATA_TOMBSTONED (TS a3d67dc22c8a4cc2ba08b836537105e7 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:38120
I20240501 14:09:58.355818 19748 tablet_replica.cc:330] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: stopping tablet replica
I20240501 14:09:58.355947 19748 raft_consensus.cc:2229] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:58.356045 19748 raft_consensus.cc:2258] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:58.356709 19748 ts_tablet_manager.cc:1875] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:58.358574 19748 ts_tablet_manager.cc:1888] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 14:09:58.358631 19748 log.cc:1196] T 627fd32202d247c2af434ab8be152102 P a3d67dc22c8a4cc2ba08b836537105e7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/627fd32202d247c2af434ab8be152102
I20240501 14:09:58.359019 12738 catalog_manager.cc:4830] TS a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): tablet 627fd32202d247c2af434ab8be152102 (table rebalance_test_table_6 [id=92a2b85a37334b65b68b262891115ec0]) successfully deleted
I20240501 14:09:58.372349 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 627fd32202d247c2af434ab8be152102 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:58.701758 14797 consensus_queue.cc:237] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } }
I20240501 14:09:58.702950 12895 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:58.702956 14927 raft_consensus.cc:1261] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Refusing update from remote peer 18bef23ddc624bd784ea38fc6ef00040: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:09:58.703346 19577 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:58.703553 19909 consensus_queue.cc:1035] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:09:58.704924 18993 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 18bef23ddc624bd784ea38fc6ef00040 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:58.705286 14927 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P 70c63e471e5b482b8b257c3e7543feb5 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:58.705361 12895 raft_consensus.cc:2932] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:58.707437 12738 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 73407db0172f454baf2bb44a85add182 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:58.707993 12749 catalog_manager.cc:5484] T 73407db0172f454baf2bb44a85add182 P a3d67dc22c8a4cc2ba08b836537105e7 reported cstate change: config changed from index 8 to 10, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 2 leader_uuid: "18bef23ddc624bd784ea38fc6ef00040" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "18bef23ddc624bd784ea38fc6ef00040" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 38509 } attrs { promote: false } } peers { permanent_uuid: "70c63e471e5b482b8b257c3e7543feb5" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 38347 } attrs { promote: false } } }
I20240501 14:09:58.710304 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 73407db0172f454baf2bb44a85add182 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:58.711273 20060 tablet_replica.cc:330] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:58.711458 20060 raft_consensus.cc:2229] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:58.711568 20060 raft_consensus.cc:2258] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:58.712100 20060 ts_tablet_manager.cc:1875] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:58.714195 20060 ts_tablet_manager.cc:1888] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 14:09:58.714263 20060 log.cc:1196] T 73407db0172f454baf2bb44a85add182 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/73407db0172f454baf2bb44a85add182
I20240501 14:09:58.714707 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet 73407db0172f454baf2bb44a85add182 (table rebalance_test_table_7 [id=9808ff3de84642bba3f2620cf19c2210]) successfully deleted
I20240501 14:09:59.254964 19814 tablet_copy_client.cc:523] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:59.256567 19814 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:59.275805 19814 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:59.276050 19814 tablet_bootstrap.cc:492] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:59.276199 19814 ts_tablet_manager.cc:1367] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.020s user 0.019s sys 0.000s
I20240501 14:09:59.276535 19814 raft_consensus.cc:348] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.276636 19814 raft_consensus.cc:729] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:59.276726 19814 consensus_queue.cc:260] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.277004 19814 ts_tablet_manager.cc:1398] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:09:59.277449 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-bdfa21a72f85470396cc525fba5e379d received from {username='slave'} at 127.6.153.201:36143
I20240501 14:09:59.277525 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-bdfa21a72f85470396cc525fba5e379d on tablet bdfa21a72f85470396cc525fba5e379d with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:59.292376 19835 tablet_copy_client.cc:523] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:59.293567 19835 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:59.302595 19840 tablet_copy_client.cc:523] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:59.303608 19840 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:59.306561 19845 tablet_copy_client.cc:523] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:59.306591 16003 raft_consensus.cc:1203] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:59.307716 19845 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:59.314332 19835 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:59.314572 19835 tablet_bootstrap.cc:492] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:59.314894 19835 ts_tablet_manager.cc:1367] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.021s user 0.021s sys 0.000s
I20240501 14:09:59.315435 19835 raft_consensus.cc:348] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.315553 19835 raft_consensus.cc:729] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:59.315639 19835 consensus_queue.cc:260] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.316051 19835 ts_tablet_manager.cc:1398] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:59.316618 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-6ac389d29e274b15b5f3db598a04f5e7 received from {username='slave'} at 127.6.153.201:36143
I20240501 14:09:59.316700 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-6ac389d29e274b15b5f3db598a04f5e7 on tablet 6ac389d29e274b15b5f3db598a04f5e7 with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:59.327518 19840 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:59.327737 19840 tablet_bootstrap.cc:492] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:59.327872 19840 ts_tablet_manager.cc:1367] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.024s user 0.018s sys 0.004s
I20240501 14:09:59.328227 19840 raft_consensus.cc:348] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.328332 19840 raft_consensus.cc:729] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:59.328426 19840 consensus_queue.cc:260] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.328743 19840 ts_tablet_manager.cc:1398] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:09:59.329188 13312 tablet_copy_service.cc:342] P 879199fbf918416cb23b0376dde0aa6e: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-15661c749d7a4cd688d4d666a5eca1c8 received from {username='slave'} at 127.6.153.201:45235
I20240501 14:09:59.329264 13312 tablet_copy_service.cc:433] P 879199fbf918416cb23b0376dde0aa6e: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-15661c749d7a4cd688d4d666a5eca1c8 on tablet 15661c749d7a4cd688d4d666a5eca1c8 with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:59.334882 19845 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:59.335072 19845 tablet_bootstrap.cc:492] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:59.335198 19845 ts_tablet_manager.cc:1367] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.028s user 0.019s sys 0.004s
I20240501 14:09:59.335561 19845 raft_consensus.cc:348] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.335640 19845 raft_consensus.cc:729] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:59.335714 19845 consensus_queue.cc:260] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.335925 19845 ts_tablet_manager.cc:1398] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:09:59.336366 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-f63509c2039d49f09eee28a45e03ad81 received from {username='slave'} at 127.6.153.201:36143
I20240501 14:09:59.336443 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-f63509c2039d49f09eee28a45e03ad81 on tablet f63509c2039d49f09eee28a45e03ad81 with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:59.364902 19850 tablet_copy_client.cc:523] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:09:59.366109 19850 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: Bootstrap starting.
I20240501 14:09:59.384006 19850 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:09:59.384238 19850 tablet_bootstrap.cc:492] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: Bootstrap complete.
I20240501 14:09:59.384397 19850 ts_tablet_manager.cc:1367] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.002s
I20240501 14:09:59.384691 19850 raft_consensus.cc:348] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.384778 19850 raft_consensus.cc:729] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 777595e1371c421fb350f0bfbb7a8035, State: Initialized, Role: LEARNER
I20240501 14:09:59.384850 19850 consensus_queue.cc:260] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: true } }
I20240501 14:09:59.385064 19850 ts_tablet_manager.cc:1398] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:09:59.385488 13181 tablet_copy_service.cc:342] P b6ea309b0a4a44dc9415766897152739: Request end of tablet copy session 777595e1371c421fb350f0bfbb7a8035-c2affe37c7174ff1be13ba4807d1fffe received from {username='slave'} at 127.6.153.201:36143
I20240501 14:09:59.385568 13181 tablet_copy_service.cc:433] P b6ea309b0a4a44dc9415766897152739: ending tablet copy session 777595e1371c421fb350f0bfbb7a8035-c2affe37c7174ff1be13ba4807d1fffe on tablet c2affe37c7174ff1be13ba4807d1fffe with peer 777595e1371c421fb350f0bfbb7a8035
I20240501 14:09:59.411808 16003 raft_consensus.cc:1203] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:59.496915 16003 raft_consensus.cc:1203] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:59.703944 16003 raft_consensus.cc:1203] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:59.770236 19017 raft_consensus.cc:1050] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:09:59.770606 19017 consensus_queue.cc:237] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:59.771785 16003 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.771939 13029 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.772118 15059 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.772171 20223 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.772343 20225 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.772567 19017 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.774751 19017 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.775240 16003 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.777020 15059 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.777364 13029 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.779174 12746 catalog_manager.cc:5484] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.783393 13160 consensus_queue.cc:237] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:59.784698 15059 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:59.784700 16003 raft_consensus.cc:1261] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:59.785027 20225 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:59.785176 20223 consensus_queue.cc:1035] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:59.786573 20225 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.787058 15059 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.787075 16003 raft_consensus.cc:2932] T bdfa21a72f85470396cc525fba5e379d P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.788383 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bdfa21a72f85470396cc525fba5e379d with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:59.789417 12749 catalog_manager.cc:5484] T bdfa21a72f85470396cc525fba5e379d P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.791553 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet bdfa21a72f85470396cc525fba5e379d with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:54350
I20240501 14:09:59.791918 20233 tablet_replica.cc:330] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:09:59.792027 20233 raft_consensus.cc:2229] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:59.792171 20233 raft_consensus.cc:2258] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:59.792656 20233 ts_tablet_manager.cc:1875] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:59.794281 20233 ts_tablet_manager.cc:1888] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:59.794343 20233 log.cc:1196] T bdfa21a72f85470396cc525fba5e379d P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/bdfa21a72f85470396cc525fba5e379d
I20240501 14:09:59.794744 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet bdfa21a72f85470396cc525fba5e379d (table rebalance_test_table_3 [id=8240fa2b5e724925b7c15130e8723b4b]) successfully deleted
I20240501 14:09:59.865972 19017 raft_consensus.cc:1050] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:09:59.866338 19017 consensus_queue.cc:237] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:59.867725 16003 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.867807 13292 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.868142 20223 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.868333 20225 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.868360 15059 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.868683 20225 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.870492 20223 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.870895 13292 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.872016 15059 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.872316 16002 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.874069 12747 catalog_manager.cc:5484] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.876971 13160 consensus_queue.cc:237] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:59.877938 16002 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:59.877938 15059 raft_consensus.cc:1261] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:09:59.878252 18688 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:59.878422 20225 consensus_queue.cc:1035] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:09:59.879696 20225 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.880239 15059 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.880331 16002 raft_consensus.cc:2932] T 6ac389d29e274b15b5f3db598a04f5e7 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.881055 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6ac389d29e274b15b5f3db598a04f5e7 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:09:59.882365 12746 catalog_manager.cc:5484] T 6ac389d29e274b15b5f3db598a04f5e7 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 8 to 9, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:59.884423 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac389d29e274b15b5f3db598a04f5e7 with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:56842
I20240501 14:09:59.884725 20238 tablet_replica.cc:330] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:09:59.884876 20238 raft_consensus.cc:2229] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:09:59.885077 20238 raft_consensus.cc:2258] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:09:59.885249 16002 raft_consensus.cc:1203] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 14:09:59.885710 20238 ts_tablet_manager.cc:1875] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:09:59.887650 20238 ts_tablet_manager.cc:1888] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:09:59.887708 20238 log.cc:1196] T 6ac389d29e274b15b5f3db598a04f5e7 P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/6ac389d29e274b15b5f3db598a04f5e7
I20240501 14:09:59.888057 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 6ac389d29e274b15b5f3db598a04f5e7 (table rebalance_test_table_1 [id=d23731070e374cdcbb85b4ea328ebebf]) successfully deleted
I20240501 14:09:59.983848 20223 raft_consensus.cc:1050] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:09:59.984210 20223 consensus_queue.cc:237] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:09:59.985459 16002 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.985482 12895 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.985482 15059 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:09:59.985800 20225 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.985999 18688 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.986158 20223 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:09:59.988029 18688 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.989439 12895 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.990397 12749 catalog_manager.cc:5484] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:09:59.990588 15059 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:09:59.991032 16002 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.159011 13160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe"
dest_uuid: "b6ea309b0a4a44dc9415766897152739"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39630
I20240501 14:10:00.159152 13160 raft_consensus.cc:595] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:00.180567 20305 raft_consensus.cc:1050] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:10:00.181005 20305 consensus_queue.cc:237] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.182368 16002 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:00.182404 14665 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:00.182579 13029 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer 879199fbf918416cb23b0376dde0aa6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:00.182783 20304 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:00.182981 20305 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:00.183140 19344 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:00.184737 19344 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.185616 14665 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.186041 16002 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.188196 13029 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.188511 12746 catalog_manager.cc:5484] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "879199fbf918416cb23b0376dde0aa6e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.377590 13160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe"
dest_uuid: "b6ea309b0a4a44dc9415766897152739"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39662
I20240501 14:10:00.377743 13160 raft_consensus.cc:595] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:00.379241 19017 raft_consensus.cc:979] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: : Instructing follower 3025ea3ec516466fbe5c5c4337733584 to start an election
I20240501 14:10:00.379374 20225 raft_consensus.cc:1067] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Signalling peer 3025ea3ec516466fbe5c5c4337733584 to start an election
I20240501 14:10:00.379781 15059 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe"
dest_uuid: "3025ea3ec516466fbe5c5c4337733584"
from {username='slave'} at 127.6.153.195:47333
I20240501 14:10:00.379902 15059 raft_consensus.cc:482] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:10:00.379964 15059 raft_consensus.cc:3037] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:00.380990 15059 raft_consensus.cc:504] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.381506 15059 leader_election.cc:288] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [CANDIDATE]: Term 2 election: Requested vote from peers a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393), b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667), 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197)
I20240501 14:10:00.381768 12895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe" candidate_uuid: "3025ea3ec516466fbe5c5c4337733584" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a3d67dc22c8a4cc2ba08b836537105e7"
I20240501 14:10:00.381911 12895 raft_consensus.cc:3037] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:00.382355 13160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe" candidate_uuid: "3025ea3ec516466fbe5c5c4337733584" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "b6ea309b0a4a44dc9415766897152739"
I20240501 14:10:00.382434 13160 raft_consensus.cc:3032] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:10:00.382475 13160 raft_consensus.cc:729] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b6ea309b0a4a44dc9415766897152739, State: Running, Role: LEADER
I20240501 14:10:00.382611 13160 consensus_queue.cc:260] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.382895 12895 raft_consensus.cc:2454] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3025ea3ec516466fbe5c5c4337733584 in term 2.
I20240501 14:10:00.383033 13160 raft_consensus.cc:3037] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:00.383937 13160 raft_consensus.cc:2454] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3025ea3ec516466fbe5c5c4337733584 in term 2.
I20240501 14:10:00.384199 14994 leader_election.cc:302] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3025ea3ec516466fbe5c5c4337733584, a3d67dc22c8a4cc2ba08b836537105e7, b6ea309b0a4a44dc9415766897152739; no voters:
I20240501 14:10:00.384377 19903 raft_consensus.cc:2781] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:10:00.384543 19903 raft_consensus.cc:686] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 2 LEADER]: Becoming Leader. State: Replica: 3025ea3ec516466fbe5c5c4337733584, State: Running, Role: LEADER
I20240501 14:10:00.384685 19903 consensus_queue.cc:237] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.386502 12749 catalog_manager.cc:5484] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: term changed from 1 to 2, leader changed from b6ea309b0a4a44dc9415766897152739 (127.6.153.195) to 3025ea3ec516466fbe5c5c4337733584 (127.6.153.200). New cstate: current_term: 2 leader_uuid: "3025ea3ec516466fbe5c5c4337733584" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:00.387832 16002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2affe37c7174ff1be13ba4807d1fffe" candidate_uuid: "3025ea3ec516466fbe5c5c4337733584" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "777595e1371c421fb350f0bfbb7a8035"
I20240501 14:10:00.388026 16002 raft_consensus.cc:3037] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:00.389014 16002 raft_consensus.cc:2454] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3025ea3ec516466fbe5c5c4337733584 in term 2.
I20240501 14:10:00.389787 13292 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8"
dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56492
I20240501 14:10:00.389897 13292 raft_consensus.cc:595] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:00.400758 19017 raft_consensus.cc:1050] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739: attempting to promote NON_VOTER 777595e1371c421fb350f0bfbb7a8035 to VOTER
I20240501 14:10:00.401098 19017 consensus_queue.cc:237] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.402354 14665 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:00.402411 16002 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035 [term 1 LEARNER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:00.402472 13029 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:00.402707 20225 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:00.402858 20223 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:00.403052 18688 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:00.404801 20223 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.405267 13029 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.406167 14665 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.406988 12749 catalog_manager.cc:5484] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:00.408450 16002 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } attrs { replace: true } } peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.410254 13160 consensus_queue.cc:237] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.411162 16002 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:00.411568 14665 raft_consensus.cc:1261] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Refusing update from remote peer b6ea309b0a4a44dc9415766897152739: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:00.411561 18688 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:00.411890 18688 consensus_queue.cc:1035] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:00.413249 20223 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.413646 14665 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.414335 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f63509c2039d49f09eee28a45e03ad81 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:00.414399 16002 raft_consensus.cc:2932] T f63509c2039d49f09eee28a45e03ad81 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.415606 12745 catalog_manager.cc:5484] T f63509c2039d49f09eee28a45e03ad81 P b6ea309b0a4a44dc9415766897152739 reported cstate change: config changed from index 8 to 9, VOTER 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "b6ea309b0a4a44dc9415766897152739" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:00.417776 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet f63509c2039d49f09eee28a45e03ad81 with delete_type TABLET_DATA_TOMBSTONED (TS 3021737e50984ca4bb2ad151dc4776ff not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:54350
I20240501 14:10:00.418368 20343 tablet_replica.cc:330] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: stopping tablet replica
I20240501 14:10:00.418493 20343 raft_consensus.cc:2229] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:00.418640 20343 raft_consensus.cc:2258] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:00.419206 20343 ts_tablet_manager.cc:1875] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:00.420935 20343 ts_tablet_manager.cc:1888] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 14:10:00.421003 20343 log.cc:1196] T f63509c2039d49f09eee28a45e03ad81 P 3021737e50984ca4bb2ad151dc4776ff: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/f63509c2039d49f09eee28a45e03ad81
I20240501 14:10:00.421375 12740 catalog_manager.cc:4830] TS 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421): tablet f63509c2039d49f09eee28a45e03ad81 (table rebalance_test_table_4 [id=85f30cad484d4f3781c10f44e5c54336]) successfully deleted
I20240501 14:10:00.609272 13292 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8"
dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56512
I20240501 14:10:00.609398 13292 raft_consensus.cc:595] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:00.617009 20304 raft_consensus.cc:979] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: : Instructing follower 2986bfcc5da34cbca2cad1643b6e35c9 to start an election
I20240501 14:10:00.617101 20304 raft_consensus.cc:1067] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Signalling peer 2986bfcc5da34cbca2cad1643b6e35c9 to start an election
I20240501 14:10:00.617528 14665 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8"
dest_uuid: "2986bfcc5da34cbca2cad1643b6e35c9"
from {username='slave'} at 127.6.153.196:43993
I20240501 14:10:00.617707 14665 raft_consensus.cc:482] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:10:00.617769 14665 raft_consensus.cc:3037] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:00.618733 14665 raft_consensus.cc:504] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.619972 14665 leader_election.cc:288] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [CANDIDATE]: Term 2 election: Requested vote from peers 3021737e50984ca4bb2ad151dc4776ff (127.6.153.194:36421), 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309), 777595e1371c421fb350f0bfbb7a8035 (127.6.153.201:41197)
I20240501 14:10:00.620239 13292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8" candidate_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "879199fbf918416cb23b0376dde0aa6e"
I20240501 14:10:00.620340 13292 raft_consensus.cc:3032] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:10:00.620376 13292 raft_consensus.cc:729] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 879199fbf918416cb23b0376dde0aa6e, State: Running, Role: LEADER
I20240501 14:10:00.620507 13292 consensus_queue.cc:260] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.620677 13292 raft_consensus.cc:3037] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:00.621577 13292 raft_consensus.cc:2454] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2986bfcc5da34cbca2cad1643b6e35c9 in term 2.
I20240501 14:10:00.624505 13029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8" candidate_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "3021737e50984ca4bb2ad151dc4776ff"
I20240501 14:10:00.624624 13029 raft_consensus.cc:3037] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:00.625483 13029 raft_consensus.cc:2454] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2986bfcc5da34cbca2cad1643b6e35c9 in term 2.
I20240501 14:10:00.625923 14599 leader_election.cc:302] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2986bfcc5da34cbca2cad1643b6e35c9, 3021737e50984ca4bb2ad151dc4776ff, 879199fbf918416cb23b0376dde0aa6e; no voters:
I20240501 14:10:00.626080 16002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15661c749d7a4cd688d4d666a5eca1c8" candidate_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "777595e1371c421fb350f0bfbb7a8035"
I20240501 14:10:00.626163 20310 raft_consensus.cc:2781] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:10:00.626214 16002 raft_consensus.cc:3037] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:00.626331 20310 raft_consensus.cc:686] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 2 LEADER]: Becoming Leader. State: Replica: 2986bfcc5da34cbca2cad1643b6e35c9, State: Running, Role: LEADER
I20240501 14:10:00.626624 20310 consensus_queue.cc:237] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.627146 16002 raft_consensus.cc:2454] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2986bfcc5da34cbca2cad1643b6e35c9 in term 2.
I20240501 14:10:00.629196 12747 catalog_manager.cc:5484] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 reported cstate change: term changed from 1 to 2, leader changed from 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) to 2986bfcc5da34cbca2cad1643b6e35c9 (127.6.153.197). New cstate: current_term: 2 leader_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:00.870183 12895 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:10:00.870563 20328 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:00.928440 16002 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:10:00.928781 20328 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:00.931349 13160 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:10:00.931725 20393 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ea309b0a4a44dc9415766897152739" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:00.933605 15059 consensus_queue.cc:237] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:00.934649 16002 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:00.934674 12895 raft_consensus.cc:1261] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Refusing update from remote peer 3025ea3ec516466fbe5c5c4337733584: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:00.935019 20391 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:00.935148 20393 consensus_queue.cc:1035] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:00.936513 20328 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.936913 12895 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P a3d67dc22c8a4cc2ba08b836537105e7 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.937791 12740 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c2affe37c7174ff1be13ba4807d1fffe with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:00.938027 16002 raft_consensus.cc:2932] T c2affe37c7174ff1be13ba4807d1fffe P 777595e1371c421fb350f0bfbb7a8035 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:00.938647 12749 catalog_manager.cc:5484] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 reported cstate change: config changed from index 8 to 10, VOTER b6ea309b0a4a44dc9415766897152739 (127.6.153.195) evicted. New cstate: current_term: 2 leader_uuid: "3025ea3ec516466fbe5c5c4337733584" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3d67dc22c8a4cc2ba08b836537105e7" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3025ea3ec516466fbe5c5c4337733584" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 34743 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:00.940861 13141 tablet_service.cc:1514] Processing DeleteTablet for tablet c2affe37c7174ff1be13ba4807d1fffe with delete_type TABLET_DATA_TOMBSTONED (TS b6ea309b0a4a44dc9415766897152739 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:55598
I20240501 14:10:00.941157 20397 tablet_replica.cc:330] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: stopping tablet replica
I20240501 14:10:00.941308 20397 raft_consensus.cc:2229] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:00.941462 20397 raft_consensus.cc:2258] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:00.942101 20397 ts_tablet_manager.cc:1875] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:00.943868 20397 ts_tablet_manager.cc:1888] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 14:10:00.943933 20397 log.cc:1196] T c2affe37c7174ff1be13ba4807d1fffe P b6ea309b0a4a44dc9415766897152739: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/c2affe37c7174ff1be13ba4807d1fffe
I20240501 14:10:00.944332 12739 catalog_manager.cc:4830] TS b6ea309b0a4a44dc9415766897152739 (127.6.153.195:38667): tablet c2affe37c7174ff1be13ba4807d1fffe (table rebalance_test_table_5 [id=508d6c2d17cd4545b269cea8da006693]) successfully deleted
I20240501 14:10:01.025027 13292 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Refusing update from remote peer 2986bfcc5da34cbca2cad1643b6e35c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:10:01.025465 20310 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879199fbf918416cb23b0376dde0aa6e" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:01.121037 16002 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 2 FOLLOWER]: Refusing update from remote peer 2986bfcc5da34cbca2cad1643b6e35c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:10:01.121421 20310 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:01.125619 13029 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer 2986bfcc5da34cbca2cad1643b6e35c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 14:10:01.126410 20414 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:01.559124 14665 consensus_queue.cc:237] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } }
I20240501 14:10:01.560314 16002 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 2 FOLLOWER]: Refusing update from remote peer 2986bfcc5da34cbca2cad1643b6e35c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:01.560333 13029 raft_consensus.cc:1261] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Refusing update from remote peer 2986bfcc5da34cbca2cad1643b6e35c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:01.560700 20414 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:01.560930 20310 consensus_queue.cc:1035] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:01.562433 20378 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 2986bfcc5da34cbca2cad1643b6e35c9 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:01.562832 16002 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 777595e1371c421fb350f0bfbb7a8035 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:01.562913 13029 raft_consensus.cc:2932] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:01.564375 12739 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 15661c749d7a4cd688d4d666a5eca1c8 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:01.565186 12745 catalog_manager.cc:5484] T 15661c749d7a4cd688d4d666a5eca1c8 P 3021737e50984ca4bb2ad151dc4776ff reported cstate change: config changed from index 8 to 10, VOTER 879199fbf918416cb23b0376dde0aa6e (127.6.153.196) evicted. New cstate: current_term: 2 leader_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3021737e50984ca4bb2ad151dc4776ff" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 36421 } } peers { permanent_uuid: "2986bfcc5da34cbca2cad1643b6e35c9" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 37879 } attrs { promote: false } } peers { permanent_uuid: "777595e1371c421fb350f0bfbb7a8035" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 41197 } attrs { promote: false } } }
I20240501 14:10:01.567746 13271 tablet_service.cc:1514] Processing DeleteTablet for tablet 15661c749d7a4cd688d4d666a5eca1c8 with delete_type TABLET_DATA_TOMBSTONED (TS 879199fbf918416cb23b0376dde0aa6e not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:56842
I20240501 14:10:01.568053 20431 tablet_replica.cc:330] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: stopping tablet replica
I20240501 14:10:01.568179 20431 raft_consensus.cc:2229] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:01.568282 20431 raft_consensus.cc:2258] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:01.568838 20431 ts_tablet_manager.cc:1875] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:01.570537 20431 ts_tablet_manager.cc:1888] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 14:10:01.570598 20431 log.cc:1196] T 15661c749d7a4cd688d4d666a5eca1c8 P 879199fbf918416cb23b0376dde0aa6e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/15661c749d7a4cd688d4d666a5eca1c8
I20240501 14:10:01.571020 12741 catalog_manager.cc:4830] TS 879199fbf918416cb23b0376dde0aa6e (127.6.153.196:33309): tablet 15661c749d7a4cd688d4d666a5eca1c8 (table rebalance_test_table_9 [id=9b5e0d7315fa484fbacbd966ec30f659]) successfully deleted
I20240501 14:10:01.743907 13007 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:01.747421 14644 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:01.749258 14908 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:01.754619 15040 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:01.755493 15983 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:01.760886 14766 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:01.767526 13141 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:10:01.769158 13271 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:01.781812 12876 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:01.876565 14766 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:01.884058 14908 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:01.885166 13007 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:01.885720 12876 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:01.886071 15040 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:01.887027 15983 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:01.887993 13141 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:10:01.892673 14644 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:01.913161 13271 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.150821 14644 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:02.151839 14766 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.152070 13007 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.153892 14908 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.162519 15040 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.171376 15983 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:02.179503 12876 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:02.190497 13271 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.199630 13141 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:10:02.290696 14766 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.291503 14644 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:02.292012 13007 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.302244 15040 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.302332 14908 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.303284 15983 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:02.312994 13141 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:10:02.321305 13271 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.341239 12876 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:02.447700 14766 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.448673 14644 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:02.456162 15040 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.464936 15983 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:02.470309 14908 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.470533 13271 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.470935 13141 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:10:02.473681 13007 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.479486 12876 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:02.591977 15040 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.593464 14908 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.603533 14644 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:02.604799 13271 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.609068 13141 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 14:10:02.609334 12876 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:02.618322 14766 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:02.621655 13007 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:02.640273 15983 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
e1cb11679f4c4a2f9311a613a170b3d7 | 127.6.153.254:41367 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.6.153.212:45545 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
18bef23ddc624bd784ea38fc6ef00040 | 127.6.153.198:38509 | HEALTHY | <none> | 2 | 0
2986bfcc5da34cbca2cad1643b6e35c9 | 127.6.153.197:37879 | HEALTHY | <none> | 1 | 0
3021737e50984ca4bb2ad151dc4776ff | 127.6.153.194:36421 | HEALTHY | <none> | 5 | 0
3025ea3ec516466fbe5c5c4337733584 | 127.6.153.200:34743 | HEALTHY | <none> | 2 | 0
70c63e471e5b482b8b257c3e7543feb5 | 127.6.153.199:38347 | HEALTHY | <none> | 2 | 0
777595e1371c421fb350f0bfbb7a8035 | 127.6.153.201:41197 | HEALTHY | <none> | 0 | 0
879199fbf918416cb23b0376dde0aa6e | 127.6.153.196:33309 | HEALTHY | <none> | 5 | 0
a3d67dc22c8a4cc2ba08b836537105e7 | 127.6.153.193:45393 | HEALTHY | <none> | 6 | 0
b6ea309b0a4a44dc9415766897152739 | 127.6.153.195:38667 | HEALTHY | <none> | 7 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 9
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
---------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.6.153.193 | experimental | 127.6.153.193:45393
local_ip_for_outbound_sockets | 127.6.153.194 | experimental | 127.6.153.194:36421
local_ip_for_outbound_sockets | 127.6.153.195 | experimental | 127.6.153.195:38667
local_ip_for_outbound_sockets | 127.6.153.196 | experimental | 127.6.153.196:33309
local_ip_for_outbound_sockets | 127.6.153.197 | experimental | 127.6.153.197:37879
local_ip_for_outbound_sockets | 127.6.153.198 | experimental | 127.6.153.198:38509
local_ip_for_outbound_sockets | 127.6.153.199 | experimental | 127.6.153.199:38347
local_ip_for_outbound_sockets | 127.6.153.200 | experimental | 127.6.153.200:34743
local_ip_for_outbound_sockets | 127.6.153.201 | experimental | 127.6.153.201:41197
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.6.153.193:45393
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.153.194:36421
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.153.195:38667
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.153.196:33309
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.6.153.197:37879
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.6.153.198:38509
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.6.153.199:38347
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.6.153.200:34743
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb | hidden | 127.6.153.201:41197
tablet_copy_download_file_inject_latency_ms | 1500 | hidden | all 9 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.6.153.212:45545 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
rebalance_test_table_0 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_5 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_6 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_7 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_8 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_9 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 10
Tablets | 30
Replicas | 90
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 14:10:02.665271 6759 log_verifier.cc:126] Checking tablet 15661c749d7a4cd688d4d666a5eca1c8
I20240501 14:10:02.690591 6759 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 15661c749d7a4cd688d4d666a5eca1c8
I20240501 14:10:02.690655 6759 log_verifier.cc:126] Checking tablet 2a19716d4a9b4acd8a3515b48d9eac32
I20240501 14:10:02.715791 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 2a19716d4a9b4acd8a3515b48d9eac32
I20240501 14:10:02.715859 6759 log_verifier.cc:126] Checking tablet 2ab8fdef27b64f87b89f6e8e4166e0c2
I20240501 14:10:02.741636 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 2ab8fdef27b64f87b89f6e8e4166e0c2
I20240501 14:10:02.741696 6759 log_verifier.cc:126] Checking tablet 2f58cd9e96244a2da7954bdaf9272b82
I20240501 14:10:02.766055 6759 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 2f58cd9e96244a2da7954bdaf9272b82
I20240501 14:10:02.766114 6759 log_verifier.cc:126] Checking tablet 34f677061e42436ab08123227f2be7bf
I20240501 14:10:02.790277 6759 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 34f677061e42436ab08123227f2be7bf
I20240501 14:10:02.790342 6759 log_verifier.cc:126] Checking tablet 37047d45ffe14570b28e459afdb66d7e
I20240501 14:10:02.814185 6759 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 37047d45ffe14570b28e459afdb66d7e
I20240501 14:10:02.814244 6759 log_verifier.cc:126] Checking tablet 3ca90427253c4320b2fea78e3fc89d92
I20240501 14:10:02.838238 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 3ca90427253c4320b2fea78e3fc89d92
I20240501 14:10:02.838302 6759 log_verifier.cc:126] Checking tablet 439e22f89e944051b44d3263c43b2fc3
I20240501 14:10:02.862169 6759 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 439e22f89e944051b44d3263c43b2fc3
I20240501 14:10:02.862226 6759 log_verifier.cc:126] Checking tablet 443ae0978f2145658d910e2ad9fa38d0
I20240501 14:10:02.886091 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 443ae0978f2145658d910e2ad9fa38d0
I20240501 14:10:02.886155 6759 log_verifier.cc:126] Checking tablet 4fc05e3adc8e4ed0b45d275725f289f4
I20240501 14:10:02.910012 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 4fc05e3adc8e4ed0b45d275725f289f4
I20240501 14:10:02.910077 6759 log_verifier.cc:126] Checking tablet 5053c0f554af4efaa221444af87dd9b9
I20240501 14:10:02.933494 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 5053c0f554af4efaa221444af87dd9b9
I20240501 14:10:02.933552 6759 log_verifier.cc:126] Checking tablet 5583f4e9fe9741649d365daa521355c8
I20240501 14:10:02.956743 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 5583f4e9fe9741649d365daa521355c8
I20240501 14:10:02.956808 6759 log_verifier.cc:126] Checking tablet 56314112ecb94098950fb77e1dd233aa
I20240501 14:10:02.979952 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 56314112ecb94098950fb77e1dd233aa
I20240501 14:10:02.980012 6759 log_verifier.cc:126] Checking tablet 627fd32202d247c2af434ab8be152102
I20240501 14:10:03.003953 6759 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 627fd32202d247c2af434ab8be152102
I20240501 14:10:03.004010 6759 log_verifier.cc:126] Checking tablet 68927b876c634175841f0859e3f535f4
I20240501 14:10:03.027479 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 68927b876c634175841f0859e3f535f4
I20240501 14:10:03.027593 6759 log_verifier.cc:126] Checking tablet 6ac389d29e274b15b5f3db598a04f5e7
I20240501 14:10:03.051816 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 6ac389d29e274b15b5f3db598a04f5e7
I20240501 14:10:03.051877 6759 log_verifier.cc:126] Checking tablet 6d6516b6f77b4389aa53862668486bba
I20240501 14:10:03.075150 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 6d6516b6f77b4389aa53862668486bba
I20240501 14:10:03.075206 6759 log_verifier.cc:126] Checking tablet 6e2d06a2ae3f4fe5a515d142d3e608e8
I20240501 14:10:03.098210 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 6e2d06a2ae3f4fe5a515d142d3e608e8
I20240501 14:10:03.098271 6759 log_verifier.cc:126] Checking tablet 73407db0172f454baf2bb44a85add182
I20240501 14:10:03.121958 6759 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 73407db0172f454baf2bb44a85add182
I20240501 14:10:03.122023 6759 log_verifier.cc:126] Checking tablet 78408b0cc50c438caf75876d8dcf88a3
I20240501 14:10:03.144996 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 78408b0cc50c438caf75876d8dcf88a3
I20240501 14:10:03.145053 6759 log_verifier.cc:126] Checking tablet 7e34fd488c214188aac9990af9de0db4
I20240501 14:10:03.168674 6759 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 7e34fd488c214188aac9990af9de0db4
I20240501 14:10:03.168733 6759 log_verifier.cc:126] Checking tablet 8a948907bfb14d0d958ebb969e5a6a50
I20240501 14:10:03.191785 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 8a948907bfb14d0d958ebb969e5a6a50
I20240501 14:10:03.191840 6759 log_verifier.cc:126] Checking tablet 920be22f0370418885ce155d9874607a
I20240501 14:10:03.215165 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 920be22f0370418885ce155d9874607a
I20240501 14:10:03.215222 6759 log_verifier.cc:126] Checking tablet 9d696c814eaa4c718e49dc32df0d6100
I20240501 14:10:03.238860 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 9d696c814eaa4c718e49dc32df0d6100
I20240501 14:10:03.238930 6759 log_verifier.cc:126] Checking tablet bdfa21a72f85470396cc525fba5e379d
I20240501 14:10:03.264863 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet bdfa21a72f85470396cc525fba5e379d
I20240501 14:10:03.264933 6759 log_verifier.cc:126] Checking tablet c2affe37c7174ff1be13ba4807d1fffe
I20240501 14:10:03.288837 6759 log_verifier.cc:177] Verified matching terms for 10 ops in tablet c2affe37c7174ff1be13ba4807d1fffe
I20240501 14:10:03.288898 6759 log_verifier.cc:126] Checking tablet d8ea7bbf6eec49728d69ec4b6e825e1e
I20240501 14:10:03.313158 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet d8ea7bbf6eec49728d69ec4b6e825e1e
I20240501 14:10:03.313213 6759 log_verifier.cc:126] Checking tablet ecd37c11bf4c45de95f861c1eb4f20d6
I20240501 14:10:03.336184 6759 log_verifier.cc:177] Verified matching terms for 6 ops in tablet ecd37c11bf4c45de95f861c1eb4f20d6
I20240501 14:10:03.336238 6759 log_verifier.cc:126] Checking tablet f63509c2039d49f09eee28a45e03ad81
I20240501 14:10:03.360430 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet f63509c2039d49f09eee28a45e03ad81
I20240501 14:10:03.360491 6759 log_verifier.cc:126] Checking tablet fe046c56dde144f9adbbbf779067476e
I20240501 14:10:03.388029 6759 log_verifier.cc:177] Verified matching terms for 9 ops in tablet fe046c56dde144f9adbbbf779067476e
I20240501 14:10:03.388547 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 12815
I20240501 14:10:03.395367 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 12947
I20240501 14:10:03.401289 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 13078
W20240501 14:10:03.402110 13225 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.153.193:45393: connect: Connection refused (error 111)
W20240501 14:10:03.404204 13225 consensus_peers.cc:485] T 6d6516b6f77b4389aa53862668486bba P 879199fbf918416cb23b0376dde0aa6e -> Peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): Couldn't send request to peer a3d67dc22c8a4cc2ba08b836537105e7. Status: Network error: Client connection negotiation failed: client connection to 127.6.153.193:45393: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:03.407986 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 13210
I20240501 14:10:03.413741 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 14582
W20240501 14:10:03.416882 14993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.153.193:45393: connect: Connection refused (error 111)
W20240501 14:10:03.417685 14993 consensus_peers.cc:485] T c2affe37c7174ff1be13ba4807d1fffe P 3025ea3ec516466fbe5c5c4337733584 -> Peer a3d67dc22c8a4cc2ba08b836537105e7 (127.6.153.193:45393): Couldn't send request to peer a3d67dc22c8a4cc2ba08b836537105e7. Status: Network error: Client connection negotiation failed: client connection to 127.6.153.193:45393: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:03.419236 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 14714
I20240501 14:10:03.424229 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 14846
I20240501 14:10:03.428694 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 14977
I20240501 14:10:03.433187 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 15796
I20240501 14:10:03.437584 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 12723
2024-05-01T14:10:03Z chronyd exiting
[ OK ] TserverAddedDuringRebalancingTest.TserverStarts/1 (28364 ms)
[----------] 1 test from TserverAddedDuringRebalancingTest (28365 ms total)
[----------] 1 test from RebalancerAndSingleReplicaTablets
[ RUN ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5
I20240501 14:10:03.524072 6759 test_util.cc:268] Using random seed: 184124387
I20240501 14:10:03.524785 6759 ts_itest-base.cc:112] Starting cluster with:
I20240501 14:10:03.524823 6759 ts_itest-base.cc:113] --------------
I20240501 14:10:03.524833 6759 ts_itest-base.cc:114] 4 tablet servers
I20240501 14:10:03.524842 6759 ts_itest-base.cc:115] 1 replicas per TS
I20240501 14:10:03.524852 6759 ts_itest-base.cc:116] --------------
2024-05-01T14:10:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T14:10:03Z Disabled control of system clock
I20240501 14:10:03.532912 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:39707
--webserver_interface=127.6.153.254
--webserver_port=0
--builtin_ntp_servers=127.6.153.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.153.254:39707
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false with env {}
W20240501 14:10:03.631531 20711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:03.631708 20711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:03.631742 20711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:03.634564 20711 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:03.634613 20711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 14:10:03.634634 20711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:03.634650 20711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 14:10:03.634665 20711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 14:10:03.638072 20711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.6.153.254:39707
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:39707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.153.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:03.638744 20711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:03.639703 20711 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:03.646492 20716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:03.646770 20717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:03.647362 20711 server_base.cc:1034] running on GCE node
W20240501 14:10:03.647567 20719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:03.647982 20711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:03.648767 20711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:03.649920 20711 hybrid_clock.cc:648] HybridClock initialized: now 1714572603649873 us; error 65 us; skew 500 ppm
I20240501 14:10:03.652041 20711 webserver.cc:460] Webserver started at http://127.6.153.254:35379/ using document root <none> and password file <none>
I20240501 14:10:03.652649 20711 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:03.652750 20711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:03.652961 20711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:03.654589 20711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "d7d4900ba14b4e5888a3b0fc2bd81530"
format_stamp: "Formatted at 2024-05-01 14:10:03 on dist-test-slave-cmf0"
I20240501 14:10:03.655025 20711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "d7d4900ba14b4e5888a3b0fc2bd81530"
format_stamp: "Formatted at 2024-05-01 14:10:03 on dist-test-slave-cmf0"
I20240501 14:10:03.657972 20711 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:10:03.659912 20725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:03.660813 20711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:03.660920 20711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
uuid: "d7d4900ba14b4e5888a3b0fc2bd81530"
format_stamp: "Formatted at 2024-05-01 14:10:03 on dist-test-slave-cmf0"
I20240501 14:10:03.660990 20711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:03.677892 20711 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:03.678414 20711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:03.678556 20711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:03.684501 20711 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.254:39707
I20240501 14:10:03.684510 20777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.254:39707 every 8 connection(s)
I20240501 14:10:03.685353 20711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 14:10:03.686758 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 20711
I20240501 14:10:03.686863 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 14:10:03.687705 20778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:03.692740 20778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530: Bootstrap starting.
I20240501 14:10:03.694739 20778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:03.695363 20778 log.cc:824] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:03.696732 20778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530: No bootstrap required, opened a new log
I20240501 14:10:03.698953 20778 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d4900ba14b4e5888a3b0fc2bd81530" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 39707 } }
I20240501 14:10:03.699102 20778 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:03.699136 20778 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d4900ba14b4e5888a3b0fc2bd81530, State: Initialized, Role: FOLLOWER
I20240501 14:10:03.699522 20778 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d4900ba14b4e5888a3b0fc2bd81530" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 39707 } }
I20240501 14:10:03.699637 20778 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:03.699693 20778 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:03.699784 20778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:03.700421 20778 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d4900ba14b4e5888a3b0fc2bd81530" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 39707 } }
I20240501 14:10:03.700695 20778 leader_election.cc:302] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7d4900ba14b4e5888a3b0fc2bd81530; no voters:
I20240501 14:10:03.700922 20778 leader_election.cc:288] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:03.701185 20783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:03.701355 20783 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [term 1 LEADER]: Becoming Leader. State: Replica: d7d4900ba14b4e5888a3b0fc2bd81530, State: Running, Role: LEADER
I20240501 14:10:03.701627 20783 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d4900ba14b4e5888a3b0fc2bd81530" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 39707 } }
I20240501 14:10:03.701859 20778 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 14:10:03.702898 20785 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7d4900ba14b4e5888a3b0fc2bd81530. Latest consensus state: current_term: 1 leader_uuid: "d7d4900ba14b4e5888a3b0fc2bd81530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d4900ba14b4e5888a3b0fc2bd81530" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 39707 } } }
I20240501 14:10:03.702944 20784 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7d4900ba14b4e5888a3b0fc2bd81530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d4900ba14b4e5888a3b0fc2bd81530" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 39707 } } }
I20240501 14:10:03.703047 20785 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [sys.catalog]: This master's current role is: LEADER
I20240501 14:10:03.703128 20784 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530 [sys.catalog]: This master's current role is: LEADER
I20240501 14:10:03.703542 20794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 14:10:03.706115 20794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 14:10:03.710496 20794 catalog_manager.cc:1347] Generated new cluster ID: d461ea96278d42f1a84adab9e2eaae6d
I20240501 14:10:03.710556 20794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 14:10:03.729151 20794 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 14:10:03.730077 20794 catalog_manager.cc:1504] Loading token signing keys...
I20240501 14:10:03.737864 20794 catalog_manager.cc:5857] T 00000000000000000000000000000000 P d7d4900ba14b4e5888a3b0fc2bd81530: Generated new TSK 0
I20240501 14:10:03.738409 20794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 14:10:03.746624 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.193:0
--local_ip_for_outbound_sockets=127.6.153.193
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:39707
--builtin_ntp_servers=127.6.153.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:10:03.845425 20802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:03.845602 20802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:03.845640 20802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:03.848539 20802 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:03.848592 20802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:03.848666 20802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.193
I20240501 14:10:03.852110 20802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:39707
--never_fsync=true
--heap_profile_path=/tmp/kudu.20802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.193
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:03.852838 20802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:03.853801 20802 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:03.861270 20810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:03.861603 20807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:03.861966 20808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:03.862424 20802 server_base.cc:1034] running on GCE node
I20240501 14:10:03.862741 20802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:03.863189 20802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:03.864327 20802 hybrid_clock.cc:648] HybridClock initialized: now 1714572603864305 us; error 35 us; skew 500 ppm
I20240501 14:10:03.866008 20802 webserver.cc:460] Webserver started at http://127.6.153.193:38683/ using document root <none> and password file <none>
I20240501 14:10:03.866469 20802 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:03.866523 20802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:03.866672 20802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:03.868108 20802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "b980ff126e8b4deba75cae8c88658f71"
format_stamp: "Formatted at 2024-05-01 14:10:03 on dist-test-slave-cmf0"
I20240501 14:10:03.868512 20802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "b980ff126e8b4deba75cae8c88658f71"
format_stamp: "Formatted at 2024-05-01 14:10:03 on dist-test-slave-cmf0"
I20240501 14:10:03.871371 20802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240501 14:10:03.873219 20816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:03.874097 20802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 14:10:03.874188 20802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "b980ff126e8b4deba75cae8c88658f71"
format_stamp: "Formatted at 2024-05-01 14:10:03 on dist-test-slave-cmf0"
I20240501 14:10:03.874262 20802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:03.888370 20802 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:03.888798 20802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:03.888937 20802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:03.889359 20802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:03.890096 20802 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:03.890141 20802 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:03.890187 20802 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:03.890219 20802 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:03.897389 20802 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.193:43091
I20240501 14:10:03.897444 20929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.193:43091 every 8 connection(s)
I20240501 14:10:03.898164 20802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 14:10:03.901415 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 20802
I20240501 14:10:03.901525 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 14:10:03.903494 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.194:0
--local_ip_for_outbound_sockets=127.6.153.194
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:39707
--builtin_ntp_servers=127.6.153.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:03.906301 20930 heartbeater.cc:347] Connected to a master server at 127.6.153.254:39707
I20240501 14:10:03.906476 20930 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:03.906924 20930 heartbeater.cc:511] Master 127.6.153.254:39707 requested a full tablet report, sending...
I20240501 14:10:03.908358 20742 ts_manager.cc:194] Registered new tserver with Master: b980ff126e8b4deba75cae8c88658f71 (127.6.153.193:43091)
I20240501 14:10:03.909901 20742 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.193:41007
W20240501 14:10:04.000782 20933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:04.000947 20933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:04.000985 20933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:04.003965 20933 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:04.004017 20933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:04.004078 20933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.194
I20240501 14:10:04.007390 20933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:39707
--never_fsync=true
--heap_profile_path=/tmp/kudu.20933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.194
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:04.008074 20933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:04.009001 20933 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:04.016170 20939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:04.016546 20941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:04.016901 20938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:04.017378 20933 server_base.cc:1034] running on GCE node
I20240501 14:10:04.017694 20933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:04.018159 20933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:04.019301 20933 hybrid_clock.cc:648] HybridClock initialized: now 1714572604019294 us; error 24 us; skew 500 ppm
I20240501 14:10:04.020860 20933 webserver.cc:460] Webserver started at http://127.6.153.194:38317/ using document root <none> and password file <none>
I20240501 14:10:04.021333 20933 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:04.021389 20933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:04.021541 20933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:04.022989 20933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.023438 20933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.026369 20933 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 14:10:04.028235 20947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.029093 20933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:04.029181 20933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.029254 20933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:04.040617 20933 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:04.041049 20933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:04.041177 20933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:04.041585 20933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:04.042337 20933 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:04.042382 20933 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.042426 20933 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:04.042455 20933 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.049804 20933 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.194:46593
I20240501 14:10:04.049854 21060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.194:46593 every 8 connection(s)
I20240501 14:10:04.050573 20933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 14:10:04.058207 21061 heartbeater.cc:347] Connected to a master server at 127.6.153.254:39707
I20240501 14:10:04.058418 21061 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:04.058404 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 20933
I20240501 14:10:04.058496 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 14:10:04.058950 21061 heartbeater.cc:511] Master 127.6.153.254:39707 requested a full tablet report, sending...
I20240501 14:10:04.059873 20741 ts_manager.cc:194] Registered new tserver with Master: 3b20f0e2ab8a4d8a80626cb321ff23d8 (127.6.153.194:46593)
I20240501 14:10:04.060514 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:0
--local_ip_for_outbound_sockets=127.6.153.195
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:39707
--builtin_ntp_servers=127.6.153.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:04.060583 20741 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.194:56505
W20240501 14:10:04.157601 21064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:04.157780 21064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:04.157819 21064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:04.160682 21064 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:04.160737 21064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:04.160797 21064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:10:04.164163 21064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:39707
--never_fsync=true
--heap_profile_path=/tmp/kudu.21064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:04.164882 21064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:04.165866 21064 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:04.173245 21072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:04.173595 21070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:04.173969 21069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:04.174404 21064 server_base.cc:1034] running on GCE node
I20240501 14:10:04.174713 21064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:04.175177 21064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:04.176327 21064 hybrid_clock.cc:648] HybridClock initialized: now 1714572604176306 us; error 33 us; skew 500 ppm
I20240501 14:10:04.177956 21064 webserver.cc:460] Webserver started at http://127.6.153.195:34655/ using document root <none> and password file <none>
I20240501 14:10:04.178452 21064 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:04.178504 21064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:04.178661 21064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:04.180127 21064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "3029f659bc7247198b18192cfe7aba88"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.180534 21064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "3029f659bc7247198b18192cfe7aba88"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.183414 21064 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:10:04.185282 21078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.186189 21064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:04.186271 21064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "3029f659bc7247198b18192cfe7aba88"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.186343 21064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:04.207790 21064 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:04.208228 21064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:04.208348 21064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:04.208730 21064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:04.209450 21064 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:04.209492 21064 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.209525 21064 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:04.209546 21064 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.216774 21064 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:38451
I20240501 14:10:04.216804 21191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:38451 every 8 connection(s)
I20240501 14:10:04.217536 21064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:10:04.224772 21192 heartbeater.cc:347] Connected to a master server at 127.6.153.254:39707
I20240501 14:10:04.224932 21192 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:04.225275 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 21064
I20240501 14:10:04.225366 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 14:10:04.225452 21192 heartbeater.cc:511] Master 127.6.153.254:39707 requested a full tablet report, sending...
I20240501 14:10:04.226437 20742 ts_manager.cc:194] Registered new tserver with Master: 3029f659bc7247198b18192cfe7aba88 (127.6.153.195:38451)
I20240501 14:10:04.227017 20742 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:47593
I20240501 14:10:04.227447 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.196:0
--local_ip_for_outbound_sockets=127.6.153.196
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:39707
--builtin_ntp_servers=127.6.153.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:10:04.323559 21195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:04.323737 21195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:04.323798 21195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:04.326725 21195 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:04.326783 21195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:04.326869 21195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.196
I20240501 14:10:04.330219 21195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:39707
--never_fsync=true
--heap_profile_path=/tmp/kudu.21195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.196
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:04.330935 21195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:04.331899 21195 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:04.339195 21201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:04.339500 21200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:04.339933 21203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:04.340371 21195 server_base.cc:1034] running on GCE node
I20240501 14:10:04.340708 21195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:04.341154 21195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:04.342316 21195 hybrid_clock.cc:648] HybridClock initialized: now 1714572604342303 us; error 27 us; skew 500 ppm
I20240501 14:10:04.343871 21195 webserver.cc:460] Webserver started at http://127.6.153.196:44343/ using document root <none> and password file <none>
I20240501 14:10:04.344337 21195 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:04.344388 21195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:04.344542 21195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:04.345953 21195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "b2836522307a4b15a7df4c7bcaf27486"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.346357 21195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "b2836522307a4b15a7df4c7bcaf27486"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.349234 21195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:04.351037 21209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.351917 21195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:04.352010 21195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "b2836522307a4b15a7df4c7bcaf27486"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:04.352083 21195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:04.362742 21195 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:04.363174 21195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:04.363303 21195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:04.363737 21195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:04.364481 21195 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:04.364523 21195 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.364557 21195 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:04.364575 21195 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:04.372041 21195 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.196:41217
I20240501 14:10:04.372066 21322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.196:41217 every 8 connection(s)
I20240501 14:10:04.372792 21195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 14:10:04.380306 21323 heartbeater.cc:347] Connected to a master server at 127.6.153.254:39707
I20240501 14:10:04.380489 21323 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:04.380925 21323 heartbeater.cc:511] Master 127.6.153.254:39707 requested a full tablet report, sending...
I20240501 14:10:04.381788 20742 ts_manager.cc:194] Registered new tserver with Master: b2836522307a4b15a7df4c7bcaf27486 (127.6.153.196:41217)
I20240501 14:10:04.381934 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 21195
I20240501 14:10:04.382028 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 14:10:04.382318 20742 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.196:52435
I20240501 14:10:04.384064 6759 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 14:10:04.391968 20742 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43578:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 14:10:04.405615 21126 tablet_service.cc:1467] Processing CreateTablet for tablet 4ade90b549e142cfaeb8bc1263fc025f (DEFAULT_TABLE table=TestTable [id=2c9060cc7d8040a193864bc0313c4f4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:04.406392 21126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ade90b549e142cfaeb8bc1263fc025f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:04.410795 21336 tablet_bootstrap.cc:492] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Bootstrap starting.
I20240501 14:10:04.412668 21336 tablet_bootstrap.cc:654] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:04.413267 21336 log.cc:824] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:04.414592 21336 tablet_bootstrap.cc:492] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: No bootstrap required, opened a new log
I20240501 14:10:04.414739 21336 ts_tablet_manager.cc:1367] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 14:10:04.417501 21336 raft_consensus.cc:348] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } }
I20240501 14:10:04.417654 21336 raft_consensus.cc:374] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:04.417698 21336 raft_consensus.cc:729] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3029f659bc7247198b18192cfe7aba88, State: Initialized, Role: FOLLOWER
I20240501 14:10:04.418041 21336 consensus_queue.cc:260] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } }
I20240501 14:10:04.418148 21336 raft_consensus.cc:388] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:04.418205 21336 raft_consensus.cc:482] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:04.418303 21336 raft_consensus.cc:3037] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:04.418953 21336 raft_consensus.cc:504] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } }
I20240501 14:10:04.419226 21336 leader_election.cc:302] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3029f659bc7247198b18192cfe7aba88; no voters:
I20240501 14:10:04.419469 21336 leader_election.cc:288] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:04.419548 21338 raft_consensus.cc:2781] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:04.419723 21338 raft_consensus.cc:686] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 1 LEADER]: Becoming Leader. State: Replica: 3029f659bc7247198b18192cfe7aba88, State: Running, Role: LEADER
I20240501 14:10:04.419981 21192 heartbeater.cc:503] Master 127.6.153.254:39707 was elected leader, sending a full tablet report...
I20240501 14:10:04.420033 21338 consensus_queue.cc:237] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } }
I20240501 14:10:04.420190 21336 ts_tablet_manager.cc:1398] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240501 14:10:04.422822 20741 catalog_manager.cc:5484] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3029f659bc7247198b18192cfe7aba88 (127.6.153.195). New cstate: current_term: 1 leader_uuid: "3029f659bc7247198b18192cfe7aba88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:04.431396 6759 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 14:10:04.432567 6759 ts_itest-base.cc:233] Waiting for 0 tablets on tserver b980ff126e8b4deba75cae8c88658f71 to finish bootstrapping
I20240501 14:10:04.437567 6759 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3b20f0e2ab8a4d8a80626cb321ff23d8 to finish bootstrapping
I20240501 14:10:04.442514 6759 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 3029f659bc7247198b18192cfe7aba88 to finish bootstrapping
I20240501 14:10:04.446450 6759 ts_itest-base.cc:233] Waiting for 0 tablets on tserver b2836522307a4b15a7df4c7bcaf27486 to finish bootstrapping
I20240501 14:10:04.450958 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 21064
I20240501 14:10:04.455667 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 21195
I20240501 14:10:04.911820 20930 heartbeater.cc:503] Master 127.6.153.254:39707 was elected leader, sending a full tablet report...
I20240501 14:10:05.062255 21061 heartbeater.cc:503] Master 127.6.153.254:39707 was elected leader, sending a full tablet report...
I20240501 14:10:08.211473 20741 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43578:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:10:08.223614 20995 tablet_service.cc:1467] Processing CreateTablet for tablet 52ab5b399a344b82beb805c6a79ec241 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3590d15f30244be5a40e00ac03fefbab]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.223637 20863 tablet_service.cc:1467] Processing CreateTablet for tablet 9a12f92edb6e4a1e90bc05d76facb8d9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3590d15f30244be5a40e00ac03fefbab]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.223626 20864 tablet_service.cc:1467] Processing CreateTablet for tablet 76bcad80a3bb42478cc6461c60e33f83 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3590d15f30244be5a40e00ac03fefbab]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.224571 20863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a12f92edb6e4a1e90bc05d76facb8d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.224571 20995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ab5b399a344b82beb805c6a79ec241. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.224911 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76bcad80a3bb42478cc6461c60e33f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.229837 21355 tablet_bootstrap.cc:492] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71: Bootstrap starting.
I20240501 14:10:08.230180 21356 tablet_bootstrap.cc:492] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Bootstrap starting.
I20240501 14:10:08.231632 21355 tablet_bootstrap.cc:654] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.231774 21356 tablet_bootstrap.cc:654] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.232262 21355 log.cc:824] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:08.232403 21356 log.cc:824] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:08.233655 21356 tablet_bootstrap.cc:492] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8: No bootstrap required, opened a new log
I20240501 14:10:08.233657 21355 tablet_bootstrap.cc:492] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71: No bootstrap required, opened a new log
I20240501 14:10:08.233793 21355 ts_tablet_manager.cc:1367] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 14:10:08.233793 21356 ts_tablet_manager.cc:1367] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 14:10:08.236524 21355 raft_consensus.cc:348] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.236524 21356 raft_consensus.cc:348] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.236668 21356 raft_consensus.cc:374] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.236672 21355 raft_consensus.cc:374] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.236697 21356 raft_consensus.cc:729] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.236722 21355 raft_consensus.cc:729] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.237048 21356 consensus_queue.cc:260] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.237048 21355 consensus_queue.cc:260] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.237154 21356 raft_consensus.cc:388] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.237157 21355 raft_consensus.cc:388] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.237212 21356 raft_consensus.cc:482] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.237215 21355 raft_consensus.cc:482] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.237300 21356 raft_consensus.cc:3037] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.237308 21355 raft_consensus.cc:3037] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.237927 21356 raft_consensus.cc:504] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.237928 21355 raft_consensus.cc:504] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.238183 21356 leader_election.cc:302] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b20f0e2ab8a4d8a80626cb321ff23d8; no voters:
I20240501 14:10:08.238190 21355 leader_election.cc:302] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b980ff126e8b4deba75cae8c88658f71; no voters:
I20240501 14:10:08.238420 21355 leader_election.cc:288] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.238420 21356 leader_election.cc:288] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.238493 21359 raft_consensus.cc:2781] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.238505 21360 raft_consensus.cc:2781] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.238652 21360 raft_consensus.cc:686] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [term 1 LEADER]: Becoming Leader. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Running, Role: LEADER
I20240501 14:10:08.238708 21359 raft_consensus.cc:686] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Running, Role: LEADER
I20240501 14:10:08.238971 21360 consensus_queue.cc:237] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.238971 21359 consensus_queue.cc:237] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.239149 21356 ts_tablet_manager.cc:1398] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240501 14:10:08.239148 21355 ts_tablet_manager.cc:1398] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20240501 14:10:08.239712 21355 tablet_bootstrap.cc:492] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71: Bootstrap starting.
I20240501 14:10:08.240535 21355 tablet_bootstrap.cc:654] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.241466 20742 catalog_manager.cc:5484] T 76bcad80a3bb42478cc6461c60e33f83 P b980ff126e8b4deba75cae8c88658f71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b980ff126e8b4deba75cae8c88658f71 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "b980ff126e8b4deba75cae8c88658f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.242470 21355 tablet_bootstrap.cc:492] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71: No bootstrap required, opened a new log
I20240501 14:10:08.242551 21355 ts_tablet_manager.cc:1367] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:10:08.242852 20741 catalog_manager.cc:5484] T 52ab5b399a344b82beb805c6a79ec241 P 3b20f0e2ab8a4d8a80626cb321ff23d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b20f0e2ab8a4d8a80626cb321ff23d8 (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.243038 21355 raft_consensus.cc:348] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.243121 21355 raft_consensus.cc:374] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.243165 21355 raft_consensus.cc:729] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.243284 21355 consensus_queue.cc:260] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.243386 21355 raft_consensus.cc:388] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.243425 21355 raft_consensus.cc:482] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.243467 21355 raft_consensus.cc:3037] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.244079 21355 raft_consensus.cc:504] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.244175 21355 leader_election.cc:302] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b980ff126e8b4deba75cae8c88658f71; no voters:
I20240501 14:10:08.244268 21355 leader_election.cc:288] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.244346 21363 raft_consensus.cc:2781] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.244446 21363 raft_consensus.cc:686] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [term 1 LEADER]: Becoming Leader. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Running, Role: LEADER
I20240501 14:10:08.244467 21355 ts_tablet_manager.cc:1398] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:08.244551 21363 consensus_queue.cc:237] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.246698 20741 catalog_manager.cc:5484] T 9a12f92edb6e4a1e90bc05d76facb8d9 P b980ff126e8b4deba75cae8c88658f71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b980ff126e8b4deba75cae8c88658f71 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "b980ff126e8b4deba75cae8c88658f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.395375 20741 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43578:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:10:08.400067 20995 tablet_service.cc:1467] Processing CreateTablet for tablet 38ccf61580824050ba178a558b476608 (DEFAULT_TABLE table=rebalance_test_table_1 [id=e71ba016b84d4a829ac83b13d900df0d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.400147 20994 tablet_service.cc:1467] Processing CreateTablet for tablet a67bd7c0d434431c8e0c97ceabc9a841 (DEFAULT_TABLE table=rebalance_test_table_1 [id=e71ba016b84d4a829ac83b13d900df0d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.400241 20864 tablet_service.cc:1467] Processing CreateTablet for tablet eb7fec80be3849c18f6e1a60300d4e14 (DEFAULT_TABLE table=rebalance_test_table_1 [id=e71ba016b84d4a829ac83b13d900df0d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.400354 20995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38ccf61580824050ba178a558b476608. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.400475 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb7fec80be3849c18f6e1a60300d4e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.400512 20994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a67bd7c0d434431c8e0c97ceabc9a841. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.402314 21356 tablet_bootstrap.cc:492] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Bootstrap starting.
I20240501 14:10:08.402493 21355 tablet_bootstrap.cc:492] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71: Bootstrap starting.
I20240501 14:10:08.403148 21356 tablet_bootstrap.cc:654] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.403316 21355 tablet_bootstrap.cc:654] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.404237 21356 tablet_bootstrap.cc:492] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8: No bootstrap required, opened a new log
I20240501 14:10:08.404301 21356 ts_tablet_manager.cc:1367] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:08.404367 21355 tablet_bootstrap.cc:492] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71: No bootstrap required, opened a new log
I20240501 14:10:08.404443 21355 ts_tablet_manager.cc:1367] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:08.404704 21356 raft_consensus.cc:348] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.404779 21356 raft_consensus.cc:374] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.404799 21356 raft_consensus.cc:729] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.404805 21355 raft_consensus.cc:348] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.404897 21355 raft_consensus.cc:374] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.404897 21356 consensus_queue.cc:260] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.404930 21355 raft_consensus.cc:729] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.404951 21356 raft_consensus.cc:388] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.404973 21356 raft_consensus.cc:482] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.405001 21356 raft_consensus.cc:3037] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.405025 21355 consensus_queue.cc:260] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.405077 21355 raft_consensus.cc:388] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.405097 21355 raft_consensus.cc:482] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.405123 21355 raft_consensus.cc:3037] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.405668 21356 raft_consensus.cc:504] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.405764 21356 leader_election.cc:302] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b20f0e2ab8a4d8a80626cb321ff23d8; no voters:
I20240501 14:10:08.405731 21355 raft_consensus.cc:504] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.405826 21356 leader_election.cc:288] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.405846 21355 leader_election.cc:302] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b980ff126e8b4deba75cae8c88658f71; no voters:
I20240501 14:10:08.405887 21359 raft_consensus.cc:2781] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.405978 21359 raft_consensus.cc:686] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Running, Role: LEADER
I20240501 14:10:08.406080 21359 consensus_queue.cc:237] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.406148 21356 ts_tablet_manager.cc:1398] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240501 14:10:08.406217 21363 raft_consensus.cc:2781] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.406301 21363 raft_consensus.cc:686] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [term 1 LEADER]: Becoming Leader. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Running, Role: LEADER
I20240501 14:10:08.406365 21356 tablet_bootstrap.cc:492] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Bootstrap starting.
I20240501 14:10:08.406415 21363 consensus_queue.cc:237] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.407151 21355 leader_election.cc:288] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.407611 21356 tablet_bootstrap.cc:654] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.407855 20742 catalog_manager.cc:5484] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b980ff126e8b4deba75cae8c88658f71 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "b980ff126e8b4deba75cae8c88658f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.407855 20741 catalog_manager.cc:5484] T a67bd7c0d434431c8e0c97ceabc9a841 P 3b20f0e2ab8a4d8a80626cb321ff23d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b20f0e2ab8a4d8a80626cb321ff23d8 (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.408232 21355 ts_tablet_manager.cc:1398] T eb7fec80be3849c18f6e1a60300d4e14 P b980ff126e8b4deba75cae8c88658f71: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240501 14:10:08.408596 21356 tablet_bootstrap.cc:492] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8: No bootstrap required, opened a new log
I20240501 14:10:08.408677 21356 ts_tablet_manager.cc:1367] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:08.409181 21356 raft_consensus.cc:348] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.409272 21356 raft_consensus.cc:374] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.409309 21356 raft_consensus.cc:729] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.409408 21356 consensus_queue.cc:260] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.409478 21356 raft_consensus.cc:388] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.409507 21356 raft_consensus.cc:482] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.409540 21356 raft_consensus.cc:3037] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.410215 21356 raft_consensus.cc:504] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.410315 21356 leader_election.cc:302] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b20f0e2ab8a4d8a80626cb321ff23d8; no voters:
I20240501 14:10:08.410393 21356 leader_election.cc:288] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.410435 21359 raft_consensus.cc:2781] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.410514 21359 raft_consensus.cc:686] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Running, Role: LEADER
I20240501 14:10:08.410574 21356 ts_tablet_manager.cc:1398] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:08.410609 21359 consensus_queue.cc:237] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.411769 20742 catalog_manager.cc:5484] T 38ccf61580824050ba178a558b476608 P 3b20f0e2ab8a4d8a80626cb321ff23d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b20f0e2ab8a4d8a80626cb321ff23d8 (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.555562 20740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43578:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:10:08.560467 20995 tablet_service.cc:1467] Processing CreateTablet for tablet 18010ef2e29043518d0199d7d43ecf92 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5f7d0e16702b4849b36fb8de8f794d30]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.560526 20863 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5f7d0e16702b4849b36fb8de8f794d30]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.560497 20864 tablet_service.cc:1467] Processing CreateTablet for tablet 07a758319a2b43c8a67bc180ef110377 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5f7d0e16702b4849b36fb8de8f794d30]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.560767 20995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18010ef2e29043518d0199d7d43ecf92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.560792 20863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5e4b6c6a3b4ab8b59f1277d6fa4af5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.560907 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a758319a2b43c8a67bc180ef110377. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.562332 21355 tablet_bootstrap.cc:492] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71: Bootstrap starting.
I20240501 14:10:08.562739 21356 tablet_bootstrap.cc:492] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Bootstrap starting.
I20240501 14:10:08.563171 21355 tablet_bootstrap.cc:654] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.563627 21356 tablet_bootstrap.cc:654] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.564432 21355 tablet_bootstrap.cc:492] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71: No bootstrap required, opened a new log
I20240501 14:10:08.564518 21355 ts_tablet_manager.cc:1367] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:08.564953 21355 raft_consensus.cc:348] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.565040 21356 tablet_bootstrap.cc:492] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8: No bootstrap required, opened a new log
I20240501 14:10:08.565037 21355 raft_consensus.cc:374] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.565089 21355 raft_consensus.cc:729] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.565107 21356 ts_tablet_manager.cc:1367] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:08.565214 21355 consensus_queue.cc:260] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.565290 21355 raft_consensus.cc:388] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.565320 21355 raft_consensus.cc:482] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.565356 21355 raft_consensus.cc:3037] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.565500 21356 raft_consensus.cc:348] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.565572 21356 raft_consensus.cc:374] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.565595 21356 raft_consensus.cc:729] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.565706 21356 consensus_queue.cc:260] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.565771 21356 raft_consensus.cc:388] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.565804 21356 raft_consensus.cc:482] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.565837 21356 raft_consensus.cc:3037] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.566110 21355 raft_consensus.cc:504] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.566232 21355 leader_election.cc:302] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b980ff126e8b4deba75cae8c88658f71; no voters:
I20240501 14:10:08.566316 21355 leader_election.cc:288] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.566401 21362 raft_consensus.cc:2781] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.566488 21362 raft_consensus.cc:686] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [term 1 LEADER]: Becoming Leader. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Running, Role: LEADER
I20240501 14:10:08.566520 21355 ts_tablet_manager.cc:1398] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:08.566587 21362 consensus_queue.cc:237] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.566588 21356 raft_consensus.cc:504] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.566673 21355 tablet_bootstrap.cc:492] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71: Bootstrap starting.
I20240501 14:10:08.566707 21356 leader_election.cc:302] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b20f0e2ab8a4d8a80626cb321ff23d8; no voters:
I20240501 14:10:08.566789 21356 leader_election.cc:288] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.566900 21361 raft_consensus.cc:2781] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.566984 21361 raft_consensus.cc:686] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Running, Role: LEADER
I20240501 14:10:08.567099 21361 consensus_queue.cc:237] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.567263 21356 ts_tablet_manager.cc:1398] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:08.568128 20742 catalog_manager.cc:5484] T 1c5e4b6c6a3b4ab8b59f1277d6fa4af5 P b980ff126e8b4deba75cae8c88658f71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b980ff126e8b4deba75cae8c88658f71 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "b980ff126e8b4deba75cae8c88658f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.568532 20741 catalog_manager.cc:5484] T 18010ef2e29043518d0199d7d43ecf92 P 3b20f0e2ab8a4d8a80626cb321ff23d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b20f0e2ab8a4d8a80626cb321ff23d8 (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.568759 21355 tablet_bootstrap.cc:654] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.569818 21355 tablet_bootstrap.cc:492] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71: No bootstrap required, opened a new log
I20240501 14:10:08.570035 21355 ts_tablet_manager.cc:1367] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:10:08.570453 21355 raft_consensus.cc:348] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.570550 21355 raft_consensus.cc:374] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.570636 21355 raft_consensus.cc:729] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.570878 21355 consensus_queue.cc:260] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.570971 21355 raft_consensus.cc:388] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.571049 21355 raft_consensus.cc:482] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.571225 21355 raft_consensus.cc:3037] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.571854 21355 raft_consensus.cc:504] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.571969 21355 leader_election.cc:302] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b980ff126e8b4deba75cae8c88658f71; no voters:
I20240501 14:10:08.572063 21355 leader_election.cc:288] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.572116 21362 raft_consensus.cc:2781] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.572201 21362 raft_consensus.cc:686] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [term 1 LEADER]: Becoming Leader. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Running, Role: LEADER
I20240501 14:10:08.572261 21355 ts_tablet_manager.cc:1398] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:08.572312 21362 consensus_queue.cc:237] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.573494 20741 catalog_manager.cc:5484] T 07a758319a2b43c8a67bc180ef110377 P b980ff126e8b4deba75cae8c88658f71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b980ff126e8b4deba75cae8c88658f71 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "b980ff126e8b4deba75cae8c88658f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.728636 20741 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43578:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 14:10:08.733429 20995 tablet_service.cc:1467] Processing CreateTablet for tablet 9602a3f3ec1c42a5996ef29d8b09014b (DEFAULT_TABLE table=rebalance_test_table_3 [id=df7bf5603ad6453ca7480b03975fdd1d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.733455 20994 tablet_service.cc:1467] Processing CreateTablet for tablet c7248a8c45bd4ce08c1eaecd2bba63fe (DEFAULT_TABLE table=rebalance_test_table_3 [id=df7bf5603ad6453ca7480b03975fdd1d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.733644 20864 tablet_service.cc:1467] Processing CreateTablet for tablet f03ee30a52ca4b918dbf935d1b7685b8 (DEFAULT_TABLE table=rebalance_test_table_3 [id=df7bf5603ad6453ca7480b03975fdd1d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:08.733752 20994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7248a8c45bd4ce08c1eaecd2bba63fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.733918 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f03ee30a52ca4b918dbf935d1b7685b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.733920 20995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9602a3f3ec1c42a5996ef29d8b09014b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:08.735374 21356 tablet_bootstrap.cc:492] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8: Bootstrap starting.
I20240501 14:10:08.735759 21355 tablet_bootstrap.cc:492] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71: Bootstrap starting.
I20240501 14:10:08.736207 21356 tablet_bootstrap.cc:654] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.736567 21355 tablet_bootstrap.cc:654] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.737180 21356 tablet_bootstrap.cc:492] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8: No bootstrap required, opened a new log
I20240501 14:10:08.737251 21356 ts_tablet_manager.cc:1367] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:08.737715 21355 tablet_bootstrap.cc:492] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71: No bootstrap required, opened a new log
I20240501 14:10:08.737668 21356 raft_consensus.cc:348] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.737764 21356 raft_consensus.cc:374] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.737792 21356 raft_consensus.cc:729] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.737803 21355 ts_tablet_manager.cc:1367] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:08.737900 21356 consensus_queue.cc:260] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.737977 21356 raft_consensus.cc:388] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.738013 21356 raft_consensus.cc:482] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.738056 21356 raft_consensus.cc:3037] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.738202 21355 raft_consensus.cc:348] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.738286 21355 raft_consensus.cc:374] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.738305 21355 raft_consensus.cc:729] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.738401 21355 consensus_queue.cc:260] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.738461 21355 raft_consensus.cc:388] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.738492 21355 raft_consensus.cc:482] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.738534 21355 raft_consensus.cc:3037] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.738739 21356 raft_consensus.cc:504] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.738852 21356 leader_election.cc:302] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b20f0e2ab8a4d8a80626cb321ff23d8; no voters:
I20240501 14:10:08.738930 21356 leader_election.cc:288] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.738972 21361 raft_consensus.cc:2781] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.739043 21361 raft_consensus.cc:686] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Running, Role: LEADER
I20240501 14:10:08.739140 21356 ts_tablet_manager.cc:1398] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:08.739177 21355 raft_consensus.cc:504] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.739149 21361 consensus_queue.cc:237] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.739300 21355 leader_election.cc:302] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b980ff126e8b4deba75cae8c88658f71; no voters:
I20240501 14:10:08.739356 21356 tablet_bootstrap.cc:492] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8: Bootstrap starting.
I20240501 14:10:08.739441 21362 raft_consensus.cc:2781] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.739522 21362 raft_consensus.cc:686] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [term 1 LEADER]: Becoming Leader. State: Replica: b980ff126e8b4deba75cae8c88658f71, State: Running, Role: LEADER
I20240501 14:10:08.739616 21362 consensus_queue.cc:237] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } }
I20240501 14:10:08.740341 21356 tablet_bootstrap.cc:654] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:08.740423 21355 leader_election.cc:288] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.740694 21355 ts_tablet_manager.cc:1398] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:10:08.740720 20741 catalog_manager.cc:5484] T c7248a8c45bd4ce08c1eaecd2bba63fe P 3b20f0e2ab8a4d8a80626cb321ff23d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b20f0e2ab8a4d8a80626cb321ff23d8 (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.740958 20742 catalog_manager.cc:5484] T f03ee30a52ca4b918dbf935d1b7685b8 P b980ff126e8b4deba75cae8c88658f71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b980ff126e8b4deba75cae8c88658f71 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "b980ff126e8b4deba75cae8c88658f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b980ff126e8b4deba75cae8c88658f71" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 43091 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.741295 21356 tablet_bootstrap.cc:492] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8: No bootstrap required, opened a new log
I20240501 14:10:08.741361 21356 ts_tablet_manager.cc:1367] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:08.741760 21356 raft_consensus.cc:348] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.741845 21356 raft_consensus.cc:374] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:08.741873 21356 raft_consensus.cc:729] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Initialized, Role: FOLLOWER
I20240501 14:10:08.741978 21356 consensus_queue.cc:260] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.742049 21356 raft_consensus.cc:388] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:08.742079 21356 raft_consensus.cc:482] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:08.742112 21356 raft_consensus.cc:3037] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:08.742816 21356 raft_consensus.cc:504] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.742930 21356 leader_election.cc:302] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b20f0e2ab8a4d8a80626cb321ff23d8; no voters:
I20240501 14:10:08.743024 21356 leader_election.cc:288] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:08.743065 21361 raft_consensus.cc:2781] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:08.743145 21361 raft_consensus.cc:686] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3b20f0e2ab8a4d8a80626cb321ff23d8, State: Running, Role: LEADER
I20240501 14:10:08.743212 21356 ts_tablet_manager.cc:1398] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240501 14:10:08.743247 21361 consensus_queue.cc:237] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } }
I20240501 14:10:08.744385 20742 catalog_manager.cc:5484] T 9602a3f3ec1c42a5996ef29d8b09014b P 3b20f0e2ab8a4d8a80626cb321ff23d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b20f0e2ab8a4d8a80626cb321ff23d8 (127.6.153.194). New cstate: current_term: 1 leader_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20f0e2ab8a4d8a80626cb321ff23d8" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 46593 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:08.886713 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:38451
--local_ip_for_outbound_sockets=127.6.153.195
--tserver_master_addrs=127.6.153.254:39707
--webserver_port=34655
--webserver_interface=127.6.153.195
--builtin_ntp_servers=127.6.153.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:10:08.989928 21461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:08.990104 21461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:08.990156 21461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:08.993142 21461 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:08.993201 21461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:08.993283 21461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:10:08.996655 21461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:38451
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=34655
--tserver_master_addrs=127.6.153.254:39707
--never_fsync=true
--heap_profile_path=/tmp/kudu.21461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:08.997342 21461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:08.998281 21461 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:09.005872 21466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:09.006147 21467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:09.006517 21469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:09.006882 21461 server_base.cc:1034] running on GCE node
I20240501 14:10:09.007191 21461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:09.007674 21461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:09.008813 21461 hybrid_clock.cc:648] HybridClock initialized: now 1714572609008795 us; error 26 us; skew 500 ppm
I20240501 14:10:09.010352 21461 webserver.cc:460] Webserver started at http://127.6.153.195:34655/ using document root <none> and password file <none>
I20240501 14:10:09.010831 21461 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:09.010901 21461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:09.013818 21461 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:09.015513 21475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:09.016378 21461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:09.016469 21461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "3029f659bc7247198b18192cfe7aba88"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:09.016814 21461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:09.036223 21461 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:09.036661 21461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:09.036792 21461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:09.037214 21461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:09.038125 21482 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240501 14:10:09.040726 21461 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240501 14:10:09.040776 21461 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20240501 14:10:09.040836 21461 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240501 14:10:09.042090 21461 ts_tablet_manager.cc:590] Registered 1 tablets
I20240501 14:10:09.042129 21461 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:09.042376 21482 tablet_bootstrap.cc:492] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Bootstrap starting.
I20240501 14:10:09.050782 21461 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:38451
I20240501 14:10:09.051102 21589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:38451 every 8 connection(s)
I20240501 14:10:09.051816 21461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:10:09.055645 21482 log.cc:824] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:09.057801 21482 tablet_bootstrap.cc:492] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:09.058135 21482 tablet_bootstrap.cc:492] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Bootstrap complete.
I20240501 14:10:09.059557 21482 ts_tablet_manager.cc:1367] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.013s
I20240501 14:10:09.061084 21590 heartbeater.cc:347] Connected to a master server at 127.6.153.254:39707
I20240501 14:10:09.061286 21590 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:09.061525 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 21461
I20240501 14:10:09.061874 21590 heartbeater.cc:511] Master 127.6.153.254:39707 requested a full tablet report, sending...
I20240501 14:10:09.061957 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.196:41217
--local_ip_for_outbound_sockets=127.6.153.196
--tserver_master_addrs=127.6.153.254:39707
--webserver_port=44343
--webserver_interface=127.6.153.196
--builtin_ntp_servers=127.6.153.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:09.062331 21482 raft_consensus.cc:348] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } }
I20240501 14:10:09.062676 21482 raft_consensus.cc:729] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3029f659bc7247198b18192cfe7aba88, State: Initialized, Role: FOLLOWER
I20240501 14:10:09.063098 21482 consensus_queue.cc:260] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } }
I20240501 14:10:09.063201 21482 raft_consensus.cc:388] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:09.063238 21482 raft_consensus.cc:482] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:09.063305 21482 raft_consensus.cc:3037] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:09.064209 20741 ts_manager.cc:194] Re-registered known tserver with Master: 3029f659bc7247198b18192cfe7aba88 (127.6.153.195:38451)
I20240501 14:10:09.065132 20741 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:57481
I20240501 14:10:09.065246 21482 raft_consensus.cc:504] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } }
I20240501 14:10:09.065521 21482 leader_election.cc:302] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3029f659bc7247198b18192cfe7aba88; no voters:
I20240501 14:10:09.065737 21482 leader_election.cc:288] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [CANDIDATE]: Term 2 election: Requested vote from peers
I20240501 14:10:09.066445 21482 ts_tablet_manager.cc:1398] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20240501 14:10:09.067224 21590 heartbeater.cc:503] Master 127.6.153.254:39707 was elected leader, sending a full tablet report...
I20240501 14:10:09.071465 21596 raft_consensus.cc:2781] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:10:09.071632 21596 raft_consensus.cc:686] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [term 2 LEADER]: Becoming Leader. State: Replica: 3029f659bc7247198b18192cfe7aba88, State: Running, Role: LEADER
I20240501 14:10:09.071889 21596 consensus_queue.cc:237] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } }
I20240501 14:10:09.073823 20741 catalog_manager.cc:5484] T 4ade90b549e142cfaeb8bc1263fc025f P 3029f659bc7247198b18192cfe7aba88 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "3029f659bc7247198b18192cfe7aba88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3029f659bc7247198b18192cfe7aba88" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 38451 } health_report { overall_health: HEALTHY } } }
W20240501 14:10:09.163147 21594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:09.163316 21594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:09.163372 21594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:09.166262 21594 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:09.166316 21594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:09.166378 21594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.196
I20240501 14:10:09.169711 21594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.196:41217
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.153.196
--webserver_port=44343
--tserver_master_addrs=127.6.153.254:39707
--never_fsync=true
--heap_profile_path=/tmp/kudu.21594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.196
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:09.170351 21594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:09.171243 21594 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:09.178453 21604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:09.178730 21607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:09.179155 21605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:09.179566 21594 server_base.cc:1034] running on GCE node
I20240501 14:10:09.179876 21594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:09.180308 21594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:09.181452 21594 hybrid_clock.cc:648] HybridClock initialized: now 1714572609181439 us; error 29 us; skew 500 ppm
I20240501 14:10:09.183249 21594 webserver.cc:460] Webserver started at http://127.6.153.196:44343/ using document root <none> and password file <none>
I20240501 14:10:09.183732 21594 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:09.183794 21594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:09.186682 21594 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20240501 14:10:09.188321 21613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:09.189182 21594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:09.189272 21594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "b2836522307a4b15a7df4c7bcaf27486"
format_stamp: "Formatted at 2024-05-01 14:10:04 on dist-test-slave-cmf0"
I20240501 14:10:09.189628 21594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:09.199959 21594 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:09.200408 21594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:09.200531 21594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:09.200950 21594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:09.201727 21594 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:09.201769 21594 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:09.201805 21594 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:09.201833 21594 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:09.209170 21594 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.196:41217
I20240501 14:10:09.209240 21726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.196:41217 every 8 connection(s)
I20240501 14:10:09.210003 21594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 14:10:09.217136 21727 heartbeater.cc:347] Connected to a master server at 127.6.153.254:39707
I20240501 14:10:09.217312 21727 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:09.217730 21727 heartbeater.cc:511] Master 127.6.153.254:39707 requested a full tablet report, sending...
I20240501 14:10:09.218561 20741 ts_manager.cc:194] Re-registered known tserver with Master: b2836522307a4b15a7df4c7bcaf27486 (127.6.153.196:41217)
I20240501 14:10:09.219084 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 21594
I20240501 14:10:09.219090 20741 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.196:46275
I20240501 14:10:09.394241 21661 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:09.394445 20995 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:09.395380 21524 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:09.395773 20864 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:09.487274 21524 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:09.487468 20995 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:09.488960 21661 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:09.497218 20864 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:09.575050 21661 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:09.576838 21524 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:09.577920 20995 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:09.579721 20864 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
d7d4900ba14b4e5888a3b0fc2bd81530 | 127.6.153.254:39707 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.6.153.212:45313 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
3029f659bc7247198b18192cfe7aba88 | 127.6.153.195:38451 | HEALTHY | <none> | 1 | 0
3b20f0e2ab8a4d8a80626cb321ff23d8 | 127.6.153.194:46593 | HEALTHY | <none> | 6 | 0
b2836522307a4b15a7df4c7bcaf27486 | 127.6.153.196:41217 | HEALTHY | <none> | 0 | 0
b980ff126e8b4deba75cae8c88658f71 | 127.6.153.193:43091 | HEALTHY | <none> | 6 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 4 server(s) checked
local_ip_for_outbound_sockets | 127.6.153.193 | experimental | 127.6.153.193:43091
local_ip_for_outbound_sockets | 127.6.153.194 | experimental | 127.6.153.194:46593
local_ip_for_outbound_sockets | 127.6.153.195 | experimental | 127.6.153.195:38451
local_ip_for_outbound_sockets | 127.6.153.196 | experimental | 127.6.153.196:41217
never_fsync | true | unsafe,advanced | all 4 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 4 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 4 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
server_dump_info_format | pb | hidden | all 4 server(s) checked
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.6.153.193:43091
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.6.153.194:46593
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.6.153.195:38451
server_dump_info_path | /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.6.153.196:41217
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.6.153.212:45313 | all 4 server(s) checked
time_source | builtin | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 6
Third Quartile | 6
Maximum | 6
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 5
Tablets | 13
Replicas | 13
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 14:10:09.602895 6759 log_verifier.cc:126] Checking tablet 07a758319a2b43c8a67bc180ef110377
I20240501 14:10:09.610138 6759 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 07a758319a2b43c8a67bc180ef110377
I20240501 14:10:09.610183 6759 log_verifier.cc:126] Checking tablet 18010ef2e29043518d0199d7d43ecf92
I20240501 14:10:09.616587 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 18010ef2e29043518d0199d7d43ecf92
I20240501 14:10:09.616627 6759 log_verifier.cc:126] Checking tablet 1c5e4b6c6a3b4ab8b59f1277d6fa4af5
I20240501 14:10:09.623147 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 1c5e4b6c6a3b4ab8b59f1277d6fa4af5
I20240501 14:10:09.623186 6759 log_verifier.cc:126] Checking tablet 38ccf61580824050ba178a558b476608
I20240501 14:10:09.629807 6759 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 38ccf61580824050ba178a558b476608
I20240501 14:10:09.629848 6759 log_verifier.cc:126] Checking tablet 4ade90b549e142cfaeb8bc1263fc025f
I20240501 14:10:09.636358 6759 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 4ade90b549e142cfaeb8bc1263fc025f
I20240501 14:10:09.636400 6759 log_verifier.cc:126] Checking tablet 52ab5b399a344b82beb805c6a79ec241
I20240501 14:10:09.642975 6759 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 52ab5b399a344b82beb805c6a79ec241
I20240501 14:10:09.643016 6759 log_verifier.cc:126] Checking tablet 76bcad80a3bb42478cc6461c60e33f83
I20240501 14:10:09.650302 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 76bcad80a3bb42478cc6461c60e33f83
I20240501 14:10:09.650349 6759 log_verifier.cc:126] Checking tablet 9602a3f3ec1c42a5996ef29d8b09014b
I20240501 14:10:09.657363 6759 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 9602a3f3ec1c42a5996ef29d8b09014b
I20240501 14:10:09.657407 6759 log_verifier.cc:126] Checking tablet 9a12f92edb6e4a1e90bc05d76facb8d9
I20240501 14:10:09.663892 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 9a12f92edb6e4a1e90bc05d76facb8d9
I20240501 14:10:09.663933 6759 log_verifier.cc:126] Checking tablet a67bd7c0d434431c8e0c97ceabc9a841
I20240501 14:10:09.670756 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet a67bd7c0d434431c8e0c97ceabc9a841
I20240501 14:10:09.670804 6759 log_verifier.cc:126] Checking tablet c7248a8c45bd4ce08c1eaecd2bba63fe
I20240501 14:10:09.677547 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet c7248a8c45bd4ce08c1eaecd2bba63fe
I20240501 14:10:09.677596 6759 log_verifier.cc:126] Checking tablet eb7fec80be3849c18f6e1a60300d4e14
I20240501 14:10:09.684404 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet eb7fec80be3849c18f6e1a60300d4e14
I20240501 14:10:09.684454 6759 log_verifier.cc:126] Checking tablet f03ee30a52ca4b918dbf935d1b7685b8
I20240501 14:10:09.692483 6759 log_verifier.cc:177] Verified matching terms for 1 ops in tablet f03ee30a52ca4b918dbf935d1b7685b8
I20240501 14:10:09.692920 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 20802
I20240501 14:10:09.698594 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 20933
I20240501 14:10:09.703785 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 21461
I20240501 14:10:09.708132 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 21594
I20240501 14:10:09.712460 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 20711
2024-05-01T14:10:09Z chronyd exiting
[ OK ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5 (6216 ms)
[----------] 1 test from RebalancerAndSingleReplicaTablets (6216 ms total)
[----------] 2 tests from LocationAwareRebalancingParamTest
[ RUN ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1
I20240501 14:10:09.740226 6759 test_util.cc:268] Using random seed: 190340542
I20240501 14:10:09.740918 6759 ts_itest-base.cc:112] Starting cluster with:
I20240501 14:10:09.740950 6759 ts_itest-base.cc:113] --------------
I20240501 14:10:09.740959 6759 ts_itest-base.cc:114] 4 tablet servers
I20240501 14:10:09.740967 6759 ts_itest-base.cc:115] 2 replicas per TS
I20240501 14:10:09.740976 6759 ts_itest-base.cc:116] --------------
2024-05-01T14:10:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T14:10:09Z Disabled control of system clock
I20240501 14:10:09.748558 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:35355
--webserver_interface=127.6.153.254
--webserver_port=0
--builtin_ntp_servers=127.6.153.212:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.153.254:35355
--location_mapping_cmd=/tmp/dist-test-taskxMccU0/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240501 14:10:09.845764 21836 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 14:10:09.845968 21836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:09.845996 21836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:09.846051 21836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:09.848927 21836 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:09.848987 21836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 14:10:09.849023 21836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:09.849054 21836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 14:10:09.849076 21836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 14:10:09.852768 21836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskxMccU0/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--ipki_ca_key_size=768
--master_addresses=127.6.153.254:35355
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:35355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.153.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.21836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:09.853451 21836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:09.854394 21836 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:09.860653 21842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:09.861030 21841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:09.861410 21844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:09.861867 21836 server_base.cc:1034] running on GCE node
I20240501 14:10:09.862278 21836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:09.863039 21836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:09.864180 21836 hybrid_clock.cc:648] HybridClock initialized: now 1714572609864170 us; error 22 us; skew 500 ppm
I20240501 14:10:09.865762 21836 webserver.cc:460] Webserver started at http://127.6.153.254:36775/ using document root <none> and password file <none>
I20240501 14:10:09.866220 21836 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:09.866272 21836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:09.866431 21836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:09.867852 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2"
format_stamp: "Formatted at 2024-05-01 14:10:09 on dist-test-slave-cmf0"
I20240501 14:10:09.868265 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2"
format_stamp: "Formatted at 2024-05-01 14:10:09 on dist-test-slave-cmf0"
I20240501 14:10:09.871022 21836 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:10:09.872941 21850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:09.873792 21836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:09.873894 21836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2"
format_stamp: "Formatted at 2024-05-01 14:10:09 on dist-test-slave-cmf0"
I20240501 14:10:09.873966 21836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:09.890586 21836 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:09.891065 21836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:09.891193 21836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:09.896821 21836 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.254:35355
I20240501 14:10:09.896843 21902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.254:35355 every 8 connection(s)
I20240501 14:10:09.897595 21836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 14:10:09.899776 21903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:09.902336 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 21836
I20240501 14:10:09.902452 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 14:10:09.904320 21903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2: Bootstrap starting.
I20240501 14:10:09.906244 21903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:09.906905 21903 log.cc:824] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:09.908433 21903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2: No bootstrap required, opened a new log
I20240501 14:10:09.910965 21903 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 35355 } }
I20240501 14:10:09.911113 21903 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:09.911160 21903 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f5a1dfd37d4c60a2d17cbe93292af2, State: Initialized, Role: FOLLOWER
I20240501 14:10:09.911525 21903 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 35355 } }
I20240501 14:10:09.911633 21903 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:09.911688 21903 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:09.911787 21903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:09.912475 21903 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 35355 } }
I20240501 14:10:09.912756 21903 leader_election.cc:302] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9f5a1dfd37d4c60a2d17cbe93292af2; no voters:
I20240501 14:10:09.912979 21903 leader_election.cc:288] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:09.913272 21908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:09.913439 21908 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [term 1 LEADER]: Becoming Leader. State: Replica: f9f5a1dfd37d4c60a2d17cbe93292af2, State: Running, Role: LEADER
I20240501 14:10:09.913664 21908 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 35355 } }
I20240501 14:10:09.913899 21903 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 14:10:09.914971 21909 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 35355 } } }
I20240501 14:10:09.915084 21909 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [sys.catalog]: This master's current role is: LEADER
I20240501 14:10:09.914963 21910 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9f5a1dfd37d4c60a2d17cbe93292af2. Latest consensus state: current_term: 1 leader_uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f5a1dfd37d4c60a2d17cbe93292af2" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 35355 } } }
I20240501 14:10:09.915283 21910 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2 [sys.catalog]: This master's current role is: LEADER
I20240501 14:10:09.915951 21917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 14:10:09.918342 21917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 14:10:09.922780 21917 catalog_manager.cc:1347] Generated new cluster ID: 8a6e02c8d9594858b5bc8a79ebc0648d
I20240501 14:10:09.922835 21917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 14:10:09.933414 21917 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 14:10:09.934126 21917 catalog_manager.cc:1504] Loading token signing keys...
I20240501 14:10:09.938390 21917 catalog_manager.cc:5857] T 00000000000000000000000000000000 P f9f5a1dfd37d4c60a2d17cbe93292af2: Generated new TSK 0
I20240501 14:10:09.938889 21917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 14:10:09.943303 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.193:0
--local_ip_for_outbound_sockets=127.6.153.193
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:35355
--builtin_ntp_servers=127.6.153.212:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:10:10.041841 21927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:10.042037 21927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:10.042085 21927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:10.044986 21927 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:10.045038 21927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:10.045104 21927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.193
I20240501 14:10:10.048440 21927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:35355
--never_fsync=true
--heap_profile_path=/tmp/kudu.21927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.193
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:10.049163 21927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:10.050148 21927 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:10.058336 21933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:10.058653 21932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:10.059000 21935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:10.059409 21927 server_base.cc:1034] running on GCE node
I20240501 14:10:10.059751 21927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:10.060245 21927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:10.061406 21927 hybrid_clock.cc:648] HybridClock initialized: now 1714572610061396 us; error 26 us; skew 500 ppm
I20240501 14:10:10.063162 21927 webserver.cc:460] Webserver started at http://127.6.153.193:33725/ using document root <none> and password file <none>
I20240501 14:10:10.063696 21927 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:10.063747 21927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:10.063905 21927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:10.065403 21927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "2ddd674017424cf78901b63950999865"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.065824 21927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "2ddd674017424cf78901b63950999865"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.068789 21927 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:10.070680 21941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.071600 21927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:10:10.071695 21927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "2ddd674017424cf78901b63950999865"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.071771 21927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:10.083886 21927 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:10.084329 21927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:10.084461 21927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:10.084890 21927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:10.085638 21927 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:10.085683 21927 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.085729 21927 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:10.085757 21927 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.093230 21927 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.193:46839
I20240501 14:10:10.093258 22054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.193:46839 every 8 connection(s)
I20240501 14:10:10.094012 21927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 14:10:10.098007 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 21927
I20240501 14:10:10.098129 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 14:10:10.100083 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.194:0
--local_ip_for_outbound_sockets=127.6.153.194
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:35355
--builtin_ntp_servers=127.6.153.212:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:10.101830 22055 heartbeater.cc:347] Connected to a master server at 127.6.153.254:35355
I20240501 14:10:10.102008 22055 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:10.102440 22055 heartbeater.cc:511] Master 127.6.153.254:35355 requested a full tablet report, sending...
I20240501 14:10:10.146200 21867 ts_manager.cc:194] Registered new tserver with Master: 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:10.147502 21867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.193:36337
W20240501 14:10:10.211164 22058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:10.211380 22058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:10.211441 22058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:10.214346 22058 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:10.214403 22058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:10.214488 22058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.194
I20240501 14:10:10.217885 22058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:35355
--never_fsync=true
--heap_profile_path=/tmp/kudu.22058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.194
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:10.218616 22058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:10.219627 22058 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:10.227006 22067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:10.227380 22064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:10.227721 22065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:10.228157 22058 server_base.cc:1034] running on GCE node
I20240501 14:10:10.228467 22058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:10.228932 22058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:10.230093 22058 hybrid_clock.cc:648] HybridClock initialized: now 1714572610230084 us; error 23 us; skew 500 ppm
I20240501 14:10:10.231719 22058 webserver.cc:460] Webserver started at http://127.6.153.194:43375/ using document root <none> and password file <none>
I20240501 14:10:10.232201 22058 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:10.232254 22058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:10.232419 22058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:10.233879 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "84b20df3357443f5b174d9beca07daad"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.234289 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "84b20df3357443f5b174d9beca07daad"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.237228 22058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240501 14:10:10.239071 22073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.240021 22058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:10.240113 22058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "84b20df3357443f5b174d9beca07daad"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.240188 22058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:10.258083 22058 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:10.258602 22058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:10.258736 22058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:10.259176 22058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:10.259999 22058 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:10.260043 22058 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.260089 22058 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:10.260118 22058 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.267829 22058 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.194:43739
I20240501 14:10:10.267853 22186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.194:43739 every 8 connection(s)
I20240501 14:10:10.268620 22058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 14:10:10.275115 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 22058
I20240501 14:10:10.275250 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 14:10:10.276321 22187 heartbeater.cc:347] Connected to a master server at 127.6.153.254:35355
I20240501 14:10:10.276505 22187 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:10.277010 22187 heartbeater.cc:511] Master 127.6.153.254:35355 requested a full tablet report, sending...
I20240501 14:10:10.277474 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:0
--local_ip_for_outbound_sockets=127.6.153.195
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:35355
--builtin_ntp_servers=127.6.153.212:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:10.307924 21867 ts_manager.cc:194] Registered new tserver with Master: 84b20df3357443f5b174d9beca07daad (127.6.153.194:43739)
I20240501 14:10:10.308614 21867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.194:34509
W20240501 14:10:10.376132 22190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:10.376345 22190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:10.376406 22190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:10.379344 22190 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:10.379410 22190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:10.379491 22190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:10:10.382885 22190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:35355
--never_fsync=true
--heap_profile_path=/tmp/kudu.22190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:10.383617 22190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:10.384567 22190 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:10.391839 22199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:10.392144 22196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:10.392474 22197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:10.392911 22190 server_base.cc:1034] running on GCE node
I20240501 14:10:10.393281 22190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:10.393715 22190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:10.394861 22190 hybrid_clock.cc:648] HybridClock initialized: now 1714572610394847 us; error 25 us; skew 500 ppm
I20240501 14:10:10.396443 22190 webserver.cc:460] Webserver started at http://127.6.153.195:44769/ using document root <none> and password file <none>
I20240501 14:10:10.396915 22190 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:10.396966 22190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:10.397122 22190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:10.398524 22190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "6ce85ae069f94132aa86b0879ad1607b"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.398937 22190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "6ce85ae069f94132aa86b0879ad1607b"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.401759 22190 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:10.403631 22205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.404495 22190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:10.404584 22190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "6ce85ae069f94132aa86b0879ad1607b"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.404657 22190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:10.430645 22190 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:10.431125 22190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:10.431258 22190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:10.431725 22190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:10.432492 22190 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:10.432534 22190 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.432579 22190 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:10.432600 22190 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.440049 22190 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:43545
I20240501 14:10:10.440081 22318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:43545 every 8 connection(s)
I20240501 14:10:10.440858 22190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:10:10.442941 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 22190
I20240501 14:10:10.443059 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 14:10:10.445135 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.196:0
--local_ip_for_outbound_sockets=127.6.153.196
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:35355
--builtin_ntp_servers=127.6.153.212:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:10.449458 22319 heartbeater.cc:347] Connected to a master server at 127.6.153.254:35355
I20240501 14:10:10.449636 22319 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:10.450062 22319 heartbeater.cc:511] Master 127.6.153.254:35355 requested a full tablet report, sending...
I20240501 14:10:10.481165 21867 ts_manager.cc:194] Registered new tserver with Master: 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195:43545)
I20240501 14:10:10.482069 21867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:52301
W20240501 14:10:10.543519 22322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:10.543686 22322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:10.543725 22322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:10.546599 22322 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:10.546653 22322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:10.546715 22322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.196
I20240501 14:10:10.550020 22322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:35355
--never_fsync=true
--heap_profile_path=/tmp/kudu.22322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.196
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:10.550688 22322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:10.551625 22322 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:10.558889 22331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:10.559235 22328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:10.559625 22329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:10.560021 22322 server_base.cc:1034] running on GCE node
I20240501 14:10:10.560326 22322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:10.560796 22322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:10.561952 22322 hybrid_clock.cc:648] HybridClock initialized: now 1714572610561927 us; error 36 us; skew 500 ppm
I20240501 14:10:10.563596 22322 webserver.cc:460] Webserver started at http://127.6.153.196:35479/ using document root <none> and password file <none>
I20240501 14:10:10.564087 22322 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:10.564142 22322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:10.564302 22322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:10.565754 22322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "5cdba816534e47eca8624ff41c9e6d9a"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.566174 22322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "5cdba816534e47eca8624ff41c9e6d9a"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.569149 22322 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:10.572988 22337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.573882 22322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:10:10.573976 22322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "5cdba816534e47eca8624ff41c9e6d9a"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:10.574052 22322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:10.598721 22322 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:10.599251 22322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:10.599412 22322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:10.599879 22322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:10.600692 22322 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:10.600736 22322 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.600785 22322 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:10.600818 22322 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:10.608438 22322 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.196:41495
I20240501 14:10:10.608498 22450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.196:41495 every 8 connection(s)
I20240501 14:10:10.609256 22322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 14:10:10.610430 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 22322
I20240501 14:10:10.610569 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 14:10:10.617347 22451 heartbeater.cc:347] Connected to a master server at 127.6.153.254:35355
I20240501 14:10:10.617512 22451 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:10.617926 22451 heartbeater.cc:511] Master 127.6.153.254:35355 requested a full tablet report, sending...
I20240501 14:10:10.648553 21867 ts_manager.cc:194] Registered new tserver with Master: 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:10.649199 21867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.196:34535
I20240501 14:10:10.659734 6759 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 14:10:10.668782 6759 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 14:10:10.668856 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 22058
I20240501 14:10:10.673483 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 22190
I20240501 14:10:11.149617 22055 heartbeater.cc:503] Master 127.6.153.254:35355 was elected leader, sending a full tablet report...
I20240501 14:10:11.651288 22451 heartbeater.cc:503] Master 127.6.153.254:35355 was elected leader, sending a full tablet report...
I20240501 14:10:14.429411 21867 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33104:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:14.432375 21867 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 14:10:14.446909 22384 tablet_service.cc:1467] Processing CreateTablet for tablet d63f4896380141cd91e024c3ca83df35 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c6701d34190b4ec3bd5e2ac772a65087]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:14.446904 21988 tablet_service.cc:1467] Processing CreateTablet for tablet d63f4896380141cd91e024c3ca83df35 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c6701d34190b4ec3bd5e2ac772a65087]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:14.447028 21989 tablet_service.cc:1467] Processing CreateTablet for tablet 7954eeebad40451898e4ac08279e2243 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c6701d34190b4ec3bd5e2ac772a65087]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:14.447468 21987 tablet_service.cc:1467] Processing CreateTablet for tablet 6a46c98a1ed44fb68c558bb034e10dc8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c6701d34190b4ec3bd5e2ac772a65087]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:14.447782 22384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63f4896380141cd91e024c3ca83df35. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:14.447860 21989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7954eeebad40451898e4ac08279e2243. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:14.446909 22385 tablet_service.cc:1467] Processing CreateTablet for tablet 7954eeebad40451898e4ac08279e2243 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c6701d34190b4ec3bd5e2ac772a65087]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:14.451300 22385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7954eeebad40451898e4ac08279e2243. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:14.451634 21987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a46c98a1ed44fb68c558bb034e10dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:14.446913 22383 tablet_service.cc:1467] Processing CreateTablet for tablet 6a46c98a1ed44fb68c558bb034e10dc8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c6701d34190b4ec3bd5e2ac772a65087]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:14.451807 22383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a46c98a1ed44fb68c558bb034e10dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:14.453572 21988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63f4896380141cd91e024c3ca83df35. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:14.453656 22468 tablet_bootstrap.cc:492] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:14.455508 22469 tablet_bootstrap.cc:492] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:14.455606 22468 tablet_bootstrap.cc:654] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:14.456355 22468 log.cc:824] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:14.457082 22469 tablet_bootstrap.cc:654] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:14.457612 22469 log.cc:824] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:14.457643 22468 tablet_bootstrap.cc:492] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:14.457769 22468 ts_tablet_manager.cc:1367] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 14:10:14.458844 22469 tablet_bootstrap.cc:492] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:14.458982 22469 ts_tablet_manager.cc:1367] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 14:10:14.460194 22468 raft_consensus.cc:348] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.460361 22468 raft_consensus.cc:374] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:14.460474 22468 raft_consensus.cc:729] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:14.460811 22468 consensus_queue.cc:260] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.461473 22469 raft_consensus.cc:348] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:14.461635 22468 ts_tablet_manager.cc:1398] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 14:10:14.461652 22469 raft_consensus.cc:374] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:14.461706 22469 raft_consensus.cc:729] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:14.461879 22468 tablet_bootstrap.cc:492] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:14.462054 22469 consensus_queue.cc:260] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:14.462697 22468 tablet_bootstrap.cc:654] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:14.462823 22469 ts_tablet_manager.cc:1398] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240501 14:10:14.463021 22469 tablet_bootstrap.cc:492] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:14.463922 22469 tablet_bootstrap.cc:654] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:14.464428 22468 tablet_bootstrap.cc:492] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:14.464512 22468 ts_tablet_manager.cc:1367] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:10:14.464988 22469 tablet_bootstrap.cc:492] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:14.465068 22469 ts_tablet_manager.cc:1367] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:14.465096 22468 raft_consensus.cc:348] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.465230 22468 raft_consensus.cc:374] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:14.465301 22468 raft_consensus.cc:729] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:14.465425 22468 consensus_queue.cc:260] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.465520 22469 raft_consensus.cc:348] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.465605 22469 raft_consensus.cc:374] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:14.465644 22469 raft_consensus.cc:729] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:14.465691 22468 ts_tablet_manager.cc:1398] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:14.465745 22469 consensus_queue.cc:260] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.465878 22468 tablet_bootstrap.cc:492] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:14.465986 22469 ts_tablet_manager.cc:1398] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:14.466179 22469 tablet_bootstrap.cc:492] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:14.466693 22468 tablet_bootstrap.cc:654] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:14.466931 22469 tablet_bootstrap.cc:654] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:14.467555 22468 tablet_bootstrap.cc:492] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:14.467634 22468 ts_tablet_manager.cc:1367] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:14.468021 22468 raft_consensus.cc:348] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:14.468114 22468 raft_consensus.cc:374] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:14.468165 22468 raft_consensus.cc:729] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:14.468281 22468 consensus_queue.cc:260] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:14.468497 22468 ts_tablet_manager.cc:1398] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:14.469105 22469 tablet_bootstrap.cc:492] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:14.469182 22469 ts_tablet_manager.cc:1367] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:10:14.469523 22469 raft_consensus.cc:348] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.469612 22469 raft_consensus.cc:374] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:14.469645 22469 raft_consensus.cc:729] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:14.469750 22469 consensus_queue.cc:260] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.469985 22469 ts_tablet_manager.cc:1398] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:14.561506 22473 raft_consensus.cc:482] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:14.561704 22473 raft_consensus.cc:504] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:14.562747 22473 leader_election.cc:288] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:14.566262 22405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a46c98a1ed44fb68c558bb034e10dc8" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a" is_pre_election: true
I20240501 14:10:14.566507 22405 raft_consensus.cc:2454] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 0.
I20240501 14:10:14.566915 21945 leader_election.cc:302] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:14.567127 22473 raft_consensus.cc:2781] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:14.567196 22473 raft_consensus.cc:482] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:14.567250 22473 raft_consensus.cc:3037] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:14.568046 22473 raft_consensus.cc:504] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:14.568277 22473 leader_election.cc:288] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Requested vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:14.568691 22405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a46c98a1ed44fb68c558bb034e10dc8" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a"
I20240501 14:10:14.568831 22405 raft_consensus.cc:3037] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:14.569950 22405 raft_consensus.cc:2454] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 1.
I20240501 14:10:14.570211 21945 leader_election.cc:302] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:14.570365 22473 raft_consensus.cc:2781] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:14.570542 22473 raft_consensus.cc:686] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Becoming Leader. State: Replica: 2ddd674017424cf78901b63950999865, State: Running, Role: LEADER
I20240501 14:10:14.570838 22473 consensus_queue.cc:237] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:14.573400 21867 catalog_manager.cc:5484] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ddd674017424cf78901b63950999865 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: UNKNOWN } } }
W20240501 14:10:14.597580 22056 tablet.cc:2369] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:10:14.612159 22452 tablet.cc:2369] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:10:14.631371 22473 raft_consensus.cc:482] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:14.631461 22473 raft_consensus.cc:504] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.631749 22473 leader_election.cc:288] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:14.632118 22405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7954eeebad40451898e4ac08279e2243" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a" is_pre_election: true
I20240501 14:10:14.632239 22405 raft_consensus.cc:2454] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 0.
I20240501 14:10:14.632496 21945 leader_election.cc:302] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:14.632655 22473 raft_consensus.cc:2781] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:14.632709 22473 raft_consensus.cc:482] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:14.632736 22473 raft_consensus.cc:3037] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:14.633445 22473 raft_consensus.cc:504] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.633666 22473 leader_election.cc:288] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Requested vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:14.634052 22405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7954eeebad40451898e4ac08279e2243" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a"
I20240501 14:10:14.634159 22405 raft_consensus.cc:3037] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:14.634819 22405 raft_consensus.cc:2454] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 1.
I20240501 14:10:14.635066 21945 leader_election.cc:302] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:14.635203 22473 raft_consensus.cc:2781] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:14.635284 22473 raft_consensus.cc:686] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Becoming Leader. State: Replica: 2ddd674017424cf78901b63950999865, State: Running, Role: LEADER
I20240501 14:10:14.635433 22473 consensus_queue.cc:237] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.636727 21867 catalog_manager.cc:5484] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ddd674017424cf78901b63950999865 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:14.832314 22473 raft_consensus.cc:482] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:14.832437 22473 raft_consensus.cc:504] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.832762 22473 leader_election.cc:288] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:14.833258 22405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a" is_pre_election: true
I20240501 14:10:14.833390 22405 raft_consensus.cc:2454] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 0.
I20240501 14:10:14.833694 21945 leader_election.cc:302] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:14.833866 22473 raft_consensus.cc:2781] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:14.833923 22473 raft_consensus.cc:482] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:14.833966 22473 raft_consensus.cc:3037] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:14.834694 22473 raft_consensus.cc:504] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.834954 22473 leader_election.cc:288] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Requested vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:14.835278 22405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a"
I20240501 14:10:14.835392 22405 raft_consensus.cc:3037] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:14.836079 22405 raft_consensus.cc:2454] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 1.
I20240501 14:10:14.836324 21945 leader_election.cc:302] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:14.836467 22473 raft_consensus.cc:2781] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:14.836550 22473 raft_consensus.cc:686] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Becoming Leader. State: Replica: 2ddd674017424cf78901b63950999865, State: Running, Role: LEADER
I20240501 14:10:14.836674 22473 consensus_queue.cc:237] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:14.838049 21867 catalog_manager.cc:5484] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ddd674017424cf78901b63950999865 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } }
W20240501 14:10:14.980821 22483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:14.981029 22483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:14.984064 22483 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 14:10:15.076272 22405 raft_consensus.cc:1261] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:15.077595 22473 consensus_queue.cc:1035] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:15.077739 22404 raft_consensus.cc:1261] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:15.077908 22405 raft_consensus.cc:1261] T 7954eeebad40451898e4ac08279e2243 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:15.078449 22473 consensus_queue.cc:1035] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:15.078711 22473 consensus_queue.cc:1035] T 7954eeebad40451898e4ac08279e2243 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:15.088575 22516 mvcc.cc:205] Tried to move back new op lower bound from 7022889431350222848 to 7022889429283926016. Current Snapshot: MvccSnapshot[applied={T|T < 7022889431350222848}]
I20240501 14:10:15.094488 22512 mvcc.cc:205] Tried to move back new op lower bound from 7022889431348355072 to 7022889429547503616. Current Snapshot: MvccSnapshot[applied={T|T < 7022889431348355072}]
INSERT report
rows total: 2000
time total: 98.7648 ms
time per row: 0.0493824 ms
I20240501 14:10:15.124550 21867 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33104:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:15.124956 21867 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 14:10:15.129218 22383 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6efade2ad54680956ecd013cd4a302 (DEFAULT_TABLE table=rebalance_test_table_1 [id=5bae9f77612c4fb09bc909d151ea4d09]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.129262 21988 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6efade2ad54680956ecd013cd4a302 (DEFAULT_TABLE table=rebalance_test_table_1 [id=5bae9f77612c4fb09bc909d151ea4d09]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.129485 22383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6efade2ad54680956ecd013cd4a302. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.129536 21988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6efade2ad54680956ecd013cd4a302. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.129695 21989 tablet_service.cc:1467] Processing CreateTablet for tablet a6d7ac45db904e25a7c57c781a5045cd (DEFAULT_TABLE table=rebalance_test_table_1 [id=5bae9f77612c4fb09bc909d151ea4d09]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.129886 21989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d7ac45db904e25a7c57c781a5045cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.129261 21987 tablet_service.cc:1467] Processing CreateTablet for tablet c3617c3fe6924c5db4aa6eada012194c (DEFAULT_TABLE table=rebalance_test_table_1 [id=5bae9f77612c4fb09bc909d151ea4d09]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.130702 21987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3617c3fe6924c5db4aa6eada012194c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.131726 22385 tablet_service.cc:1467] Processing CreateTablet for tablet a6d7ac45db904e25a7c57c781a5045cd (DEFAULT_TABLE table=rebalance_test_table_1 [id=5bae9f77612c4fb09bc909d151ea4d09]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.131762 22384 tablet_service.cc:1467] Processing CreateTablet for tablet c3617c3fe6924c5db4aa6eada012194c (DEFAULT_TABLE table=rebalance_test_table_1 [id=5bae9f77612c4fb09bc909d151ea4d09]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.131934 22385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d7ac45db904e25a7c57c781a5045cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.132081 22384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3617c3fe6924c5db4aa6eada012194c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.132629 22534 tablet_bootstrap.cc:492] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:15.133503 22534 tablet_bootstrap.cc:654] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.133556 22533 tablet_bootstrap.cc:492] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:15.134290 22533 tablet_bootstrap.cc:654] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.134490 22534 tablet_bootstrap.cc:492] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:15.134553 22534 ts_tablet_manager.cc:1367] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:15.134902 22534 raft_consensus.cc:348] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.134997 22534 raft_consensus.cc:374] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.135030 22534 raft_consensus.cc:729] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.135169 22534 consensus_queue.cc:260] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.135545 22534 ts_tablet_manager.cc:1398] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:15.135622 22533 tablet_bootstrap.cc:492] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:15.135730 22533 ts_tablet_manager.cc:1367] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:15.135944 22534 tablet_bootstrap.cc:492] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:15.136082 22533 raft_consensus.cc:348] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.136164 22533 raft_consensus.cc:374] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.136193 22533 raft_consensus.cc:729] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.136317 22533 consensus_queue.cc:260] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.136559 22533 ts_tablet_manager.cc:1398] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:15.136749 22534 tablet_bootstrap.cc:654] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.136763 22533 tablet_bootstrap.cc:492] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:15.137573 22533 tablet_bootstrap.cc:654] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.137746 22534 tablet_bootstrap.cc:492] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:15.137831 22534 ts_tablet_manager.cc:1367] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:15.138197 22534 raft_consensus.cc:348] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.138288 22534 raft_consensus.cc:374] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.138317 22534 raft_consensus.cc:729] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.138423 22534 consensus_queue.cc:260] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.138655 22534 ts_tablet_manager.cc:1398] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:15.138769 22533 tablet_bootstrap.cc:492] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:15.138798 22534 tablet_bootstrap.cc:492] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:15.138839 22533 ts_tablet_manager.cc:1367] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:15.139197 22533 raft_consensus.cc:348] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.139266 22533 raft_consensus.cc:374] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.139284 22533 raft_consensus.cc:729] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.139387 22533 consensus_queue.cc:260] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.139616 22533 ts_tablet_manager.cc:1398] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:15.139803 22533 tablet_bootstrap.cc:492] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:15.139832 22534 tablet_bootstrap.cc:654] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.140501 22533 tablet_bootstrap.cc:654] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.140854 22534 tablet_bootstrap.cc:492] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:15.140928 22534 ts_tablet_manager.cc:1367] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:15.141307 22534 raft_consensus.cc:348] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.141386 22534 raft_consensus.cc:374] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.141422 22533 tablet_bootstrap.cc:492] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:15.141438 22534 raft_consensus.cc:729] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.141484 22533 ts_tablet_manager.cc:1367] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:15.141589 22534 consensus_queue.cc:260] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.141820 22534 ts_tablet_manager.cc:1398] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:15.141816 22533 raft_consensus.cc:348] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.141896 22533 raft_consensus.cc:374] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.141924 22533 raft_consensus.cc:729] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.142019 22533 consensus_queue.cc:260] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.142247 22533 ts_tablet_manager.cc:1398] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:15.228762 22525 raft_consensus.cc:482] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:15.228881 22525 raft_consensus.cc:504] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.229223 22525 leader_election.cc:288] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:15.229609 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6efade2ad54680956ecd013cd4a302" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a" is_pre_election: true
I20240501 14:10:15.229749 22403 raft_consensus.cc:2454] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 0.
I20240501 14:10:15.230017 21945 leader_election.cc:302] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:15.230178 22525 raft_consensus.cc:2781] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:15.230237 22525 raft_consensus.cc:482] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:15.230273 22525 raft_consensus.cc:3037] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:15.231066 22525 raft_consensus.cc:504] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.231307 22525 leader_election.cc:288] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Requested vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:15.231626 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6efade2ad54680956ecd013cd4a302" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a"
I20240501 14:10:15.231731 22403 raft_consensus.cc:3037] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:15.232439 22403 raft_consensus.cc:2454] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 1.
I20240501 14:10:15.232668 21945 leader_election.cc:302] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:15.232815 22525 raft_consensus.cc:2781] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:15.232898 22525 raft_consensus.cc:686] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Becoming Leader. State: Replica: 2ddd674017424cf78901b63950999865, State: Running, Role: LEADER
I20240501 14:10:15.233023 22525 consensus_queue.cc:237] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.234364 21867 catalog_manager.cc:5484] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ddd674017424cf78901b63950999865 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:15.346065 22513 raft_consensus.cc:482] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:15.346189 22513 raft_consensus.cc:504] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.347033 22513 leader_election.cc:288] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:15.350281 22009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d7ac45db904e25a7c57c781a5045cd" candidate_uuid: "5cdba816534e47eca8624ff41c9e6d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ddd674017424cf78901b63950999865" is_pre_election: true
I20240501 14:10:15.350462 22009 raft_consensus.cc:2454] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdba816534e47eca8624ff41c9e6d9a in term 0.
I20240501 14:10:15.350792 22341 leader_election.cc:302] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:15.351009 22513 raft_consensus.cc:2781] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:15.351070 22513 raft_consensus.cc:482] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:15.351115 22513 raft_consensus.cc:3037] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:15.351913 22513 raft_consensus.cc:504] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.352152 22513 leader_election.cc:288] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 election: Requested vote from peers 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:15.352485 22009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d7ac45db904e25a7c57c781a5045cd" candidate_uuid: "5cdba816534e47eca8624ff41c9e6d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ddd674017424cf78901b63950999865"
I20240501 14:10:15.352587 22009 raft_consensus.cc:3037] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:15.353276 22009 raft_consensus.cc:2454] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdba816534e47eca8624ff41c9e6d9a in term 1.
I20240501 14:10:15.353511 22341 leader_election.cc:302] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:15.353662 22513 raft_consensus.cc:2781] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:15.353753 22513 raft_consensus.cc:686] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Becoming Leader. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Running, Role: LEADER
I20240501 14:10:15.353994 22513 consensus_queue.cc:237] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.355883 21867 catalog_manager.cc:5484] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196). New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:15.561431 22513 raft_consensus.cc:482] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:15.561573 22513 raft_consensus.cc:504] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.561967 22513 leader_election.cc:288] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:15.562424 22009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3617c3fe6924c5db4aa6eada012194c" candidate_uuid: "5cdba816534e47eca8624ff41c9e6d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ddd674017424cf78901b63950999865" is_pre_election: true
I20240501 14:10:15.562552 22009 raft_consensus.cc:2454] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdba816534e47eca8624ff41c9e6d9a in term 0.
I20240501 14:10:15.562848 22341 leader_election.cc:302] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:15.563030 22513 raft_consensus.cc:2781] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:15.563084 22513 raft_consensus.cc:482] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:15.563110 22513 raft_consensus.cc:3037] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:15.563937 22513 raft_consensus.cc:504] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.564186 22513 leader_election.cc:288] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 election: Requested vote from peers 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:15.564550 22009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3617c3fe6924c5db4aa6eada012194c" candidate_uuid: "5cdba816534e47eca8624ff41c9e6d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ddd674017424cf78901b63950999865"
I20240501 14:10:15.564659 22009 raft_consensus.cc:3037] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:15.565323 22009 raft_consensus.cc:2454] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdba816534e47eca8624ff41c9e6d9a in term 1.
I20240501 14:10:15.565577 22341 leader_election.cc:302] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:15.565726 22513 raft_consensus.cc:2781] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:15.565812 22513 raft_consensus.cc:686] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Becoming Leader. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Running, Role: LEADER
I20240501 14:10:15.565936 22513 consensus_queue.cc:237] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.567416 21867 catalog_manager.cc:5484] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196). New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:15.615124 22525 consensus_queue.cc:1035] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 14:10:15.678834 22545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:15.679035 22545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:15.681955 22545 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 14:10:15.764436 22009 raft_consensus.cc:1261] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:15.765076 22513 consensus_queue.cc:1035] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:15.765522 22009 raft_consensus.cc:1261] T c3617c3fe6924c5db4aa6eada012194c P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:15.767359 22513 consensus_queue.cc:1035] T c3617c3fe6924c5db4aa6eada012194c P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 86.4265 ms
time per row: 0.0432133 ms
I20240501 14:10:15.804083 21865 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33104:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:15.804494 21865 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 14:10:15.808833 22384 tablet_service.cc:1467] Processing CreateTablet for tablet 04e725a947374acea55ba9fc10eb7208 (DEFAULT_TABLE table=rebalance_test_table_2 [id=ed1ae7e241284581a321010797e0e9b4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.808872 21987 tablet_service.cc:1467] Processing CreateTablet for tablet 04e725a947374acea55ba9fc10eb7208 (DEFAULT_TABLE table=rebalance_test_table_2 [id=ed1ae7e241284581a321010797e0e9b4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.809103 22384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04e725a947374acea55ba9fc10eb7208. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.809149 21987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04e725a947374acea55ba9fc10eb7208. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.810757 22593 tablet_bootstrap.cc:492] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:15.810796 22594 tablet_bootstrap.cc:492] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:15.811249 22385 tablet_service.cc:1467] Processing CreateTablet for tablet 083a3f99cc90422c953adb4be0afbc99 (DEFAULT_TABLE table=rebalance_test_table_2 [id=ed1ae7e241284581a321010797e0e9b4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.811667 22594 tablet_bootstrap.cc:654] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.811726 21988 tablet_service.cc:1467] Processing CreateTablet for tablet 083a3f99cc90422c953adb4be0afbc99 (DEFAULT_TABLE table=rebalance_test_table_2 [id=ed1ae7e241284581a321010797e0e9b4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.811926 21988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 083a3f99cc90422c953adb4be0afbc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.812863 22594 tablet_bootstrap.cc:492] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:15.812916 22594 ts_tablet_manager.cc:1367] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:15.813225 22594 raft_consensus.cc:348] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.811399 22384 tablet_service.cc:1467] Processing CreateTablet for tablet 30328a0a24b543ceb69cb752fb23726a (DEFAULT_TABLE table=rebalance_test_table_2 [id=ed1ae7e241284581a321010797e0e9b4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.813308 22594 raft_consensus.cc:374] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.813338 22594 raft_consensus.cc:729] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.811859 22593 tablet_bootstrap.cc:654] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.813412 22385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 083a3f99cc90422c953adb4be0afbc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.813459 22594 consensus_queue.cc:260] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.813724 22594 ts_tablet_manager.cc:1398] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:15.813897 22384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30328a0a24b543ceb69cb752fb23726a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.813915 22594 tablet_bootstrap.cc:492] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:15.814707 22594 tablet_bootstrap.cc:654] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.815663 22594 tablet_bootstrap.cc:492] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:15.815737 22594 ts_tablet_manager.cc:1367] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:15.815975 21987 tablet_service.cc:1467] Processing CreateTablet for tablet 30328a0a24b543ceb69cb752fb23726a (DEFAULT_TABLE table=rebalance_test_table_2 [id=ed1ae7e241284581a321010797e0e9b4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:15.816059 22594 raft_consensus.cc:348] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.816175 22594 raft_consensus.cc:374] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.816190 21987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30328a0a24b543ceb69cb752fb23726a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:15.816215 22594 raft_consensus.cc:729] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.816315 22594 consensus_queue.cc:260] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.816543 22593 tablet_bootstrap.cc:492] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:15.816558 22594 ts_tablet_manager.cc:1398] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:15.816622 22593 ts_tablet_manager.cc:1367] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:10:15.817005 22593 raft_consensus.cc:348] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.817106 22593 raft_consensus.cc:374] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.817147 22593 raft_consensus.cc:729] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.817265 22593 consensus_queue.cc:260] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.817504 22593 ts_tablet_manager.cc:1398] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:15.817673 22593 tablet_bootstrap.cc:492] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:15.818423 22593 tablet_bootstrap.cc:654] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.818454 22594 tablet_bootstrap.cc:492] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865: Bootstrap starting.
I20240501 14:10:15.819186 22594 tablet_bootstrap.cc:654] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.819464 22593 tablet_bootstrap.cc:492] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:15.819542 22593 ts_tablet_manager.cc:1367] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:15.819905 22593 raft_consensus.cc:348] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.819984 22593 raft_consensus.cc:374] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.820013 22593 raft_consensus.cc:729] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.820108 22593 consensus_queue.cc:260] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:15.820336 22593 ts_tablet_manager.cc:1398] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:15.820467 22594 tablet_bootstrap.cc:492] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865: No bootstrap required, opened a new log
I20240501 14:10:15.820479 22593 tablet_bootstrap.cc:492] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: Bootstrap starting.
I20240501 14:10:15.820531 22594 ts_tablet_manager.cc:1367] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:15.820883 22594 raft_consensus.cc:348] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.820972 22594 raft_consensus.cc:374] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.821000 22594 raft_consensus.cc:729] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.821135 22594 consensus_queue.cc:260] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.821230 22593 tablet_bootstrap.cc:654] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:15.821373 22594 ts_tablet_manager.cc:1398] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:15.822119 22593 tablet_bootstrap.cc:492] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: No bootstrap required, opened a new log
I20240501 14:10:15.822199 22593 ts_tablet_manager.cc:1367] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:15.822535 22593 raft_consensus.cc:348] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.822633 22593 raft_consensus.cc:374] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:15.822651 22593 raft_consensus.cc:729] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Initialized, Role: FOLLOWER
I20240501 14:10:15.822749 22593 consensus_queue.cc:260] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.822974 22593 ts_tablet_manager.cc:1398] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:15.848367 22580 raft_consensus.cc:482] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:15.848449 22580 raft_consensus.cc:504] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.848723 22580 leader_election.cc:288] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:15.849015 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e725a947374acea55ba9fc10eb7208" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a" is_pre_election: true
I20240501 14:10:15.849112 22403 raft_consensus.cc:2454] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 0.
I20240501 14:10:15.849364 21945 leader_election.cc:302] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:15.849511 22580 raft_consensus.cc:2781] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:15.849560 22580 raft_consensus.cc:482] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:15.849604 22580 raft_consensus.cc:3037] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:15.850266 22580 raft_consensus.cc:504] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.850484 22580 leader_election.cc:288] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Requested vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:15.850759 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e725a947374acea55ba9fc10eb7208" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a"
I20240501 14:10:15.850848 22403 raft_consensus.cc:3037] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:15.851480 22403 raft_consensus.cc:2454] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 1.
I20240501 14:10:15.851711 21945 leader_election.cc:302] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:15.851848 22580 raft_consensus.cc:2781] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:15.851946 22580 raft_consensus.cc:686] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Becoming Leader. State: Replica: 2ddd674017424cf78901b63950999865, State: Running, Role: LEADER
I20240501 14:10:15.852053 22580 consensus_queue.cc:237] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:15.853266 21863 catalog_manager.cc:5484] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ddd674017424cf78901b63950999865 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:16.002166 22515 raft_consensus.cc:482] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:16.002280 22515 raft_consensus.cc:504] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:16.002614 22515 leader_election.cc:288] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:16.003023 22008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "083a3f99cc90422c953adb4be0afbc99" candidate_uuid: "5cdba816534e47eca8624ff41c9e6d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ddd674017424cf78901b63950999865" is_pre_election: true
I20240501 14:10:16.003149 22008 raft_consensus.cc:2454] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdba816534e47eca8624ff41c9e6d9a in term 0.
I20240501 14:10:16.003410 22341 leader_election.cc:302] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:16.003561 22515 raft_consensus.cc:2781] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:16.003615 22515 raft_consensus.cc:482] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:16.003652 22515 raft_consensus.cc:3037] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:16.004423 22515 raft_consensus.cc:504] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:16.004654 22515 leader_election.cc:288] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 election: Requested vote from peers 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:16.004985 22580 raft_consensus.cc:482] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:16.004998 22008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "083a3f99cc90422c953adb4be0afbc99" candidate_uuid: "5cdba816534e47eca8624ff41c9e6d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ddd674017424cf78901b63950999865"
I20240501 14:10:16.005100 22580 raft_consensus.cc:504] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:16.005144 22008 raft_consensus.cc:3037] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:16.005367 22580 leader_election.cc:288] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:16.005738 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30328a0a24b543ceb69cb752fb23726a" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a" is_pre_election: true
I20240501 14:10:16.005861 22403 raft_consensus.cc:2454] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 0.
I20240501 14:10:16.006000 22008 raft_consensus.cc:2454] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdba816534e47eca8624ff41c9e6d9a in term 1.
I20240501 14:10:16.006096 21945 leader_election.cc:302] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:16.006254 22580 raft_consensus.cc:2781] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:16.006250 22341 leader_election.cc:302] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:16.006300 22580 raft_consensus.cc:482] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:16.006325 22580 raft_consensus.cc:3037] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:16.006387 22515 raft_consensus.cc:2781] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:16.006461 22515 raft_consensus.cc:686] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Becoming Leader. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Running, Role: LEADER
I20240501 14:10:16.006582 22515 consensus_queue.cc:237] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } }
I20240501 14:10:16.007025 22580 raft_consensus.cc:504] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:16.007289 22580 leader_election.cc:288] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Requested vote from peers 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:16.007635 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30328a0a24b543ceb69cb752fb23726a" candidate_uuid: "2ddd674017424cf78901b63950999865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a"
I20240501 14:10:16.007736 22403 raft_consensus.cc:3037] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:16.007939 21863 catalog_manager.cc:5484] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196). New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:16.008342 22403 raft_consensus.cc:2454] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ddd674017424cf78901b63950999865 in term 1.
I20240501 14:10:16.008574 21945 leader_election.cc:302] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2ddd674017424cf78901b63950999865, 5cdba816534e47eca8624ff41c9e6d9a; no voters:
I20240501 14:10:16.008754 22580 raft_consensus.cc:2781] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:16.008929 22580 raft_consensus.cc:686] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Becoming Leader. State: Replica: 2ddd674017424cf78901b63950999865, State: Running, Role: LEADER
I20240501 14:10:16.009040 22580 consensus_queue.cc:237] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } }
I20240501 14:10:16.010275 21863 catalog_manager.cc:5484] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ddd674017424cf78901b63950999865 (127.6.153.193). New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } }
W20240501 14:10:16.137542 22604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:16.137751 22604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:16.140705 22604 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 14:10:16.209161 22008 raft_consensus.cc:1261] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:16.209656 22515 consensus_queue.cc:1035] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:16.210937 22403 raft_consensus.cc:1261] T 04e725a947374acea55ba9fc10eb7208 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:16.211175 22404 raft_consensus.cc:1261] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:16.211711 22580 consensus_queue.cc:1035] T 04e725a947374acea55ba9fc10eb7208 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:16.212018 22524 consensus_queue.cc:1035] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 75.1035 ms
time per row: 0.0375517 ms
I20240501 14:10:16.250942 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.194:43739
--local_ip_for_outbound_sockets=127.6.153.194
--tserver_master_addrs=127.6.153.254:35355
--webserver_port=43375
--webserver_interface=127.6.153.194
--builtin_ntp_servers=127.6.153.212:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:10:16.351689 22635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:16.351864 22635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:16.351919 22635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:16.354750 22635 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:16.354807 22635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:16.354889 22635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.194
I20240501 14:10:16.358250 22635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.194:43739
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.153.194
--webserver_port=43375
--tserver_master_addrs=127.6.153.254:35355
--never_fsync=true
--heap_profile_path=/tmp/kudu.22635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.194
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:16.358966 22635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:16.359972 22635 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:16.367321 22643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:16.367694 22640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:16.368072 22641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:16.368510 22635 server_base.cc:1034] running on GCE node
I20240501 14:10:16.368854 22635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:16.369309 22635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:16.370443 22635 hybrid_clock.cc:648] HybridClock initialized: now 1714572616370433 us; error 27 us; skew 500 ppm
I20240501 14:10:16.372011 22635 webserver.cc:460] Webserver started at http://127.6.153.194:43375/ using document root <none> and password file <none>
I20240501 14:10:16.372483 22635 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:16.372547 22635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:16.375614 22635 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:10:16.377292 22649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:16.378229 22635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:16.378343 22635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "84b20df3357443f5b174d9beca07daad"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:16.378683 22635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:16.404063 22635 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:16.404528 22635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:16.404673 22635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:16.405108 22635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:16.405886 22635 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:16.405930 22635 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:16.405968 22635 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:16.405997 22635 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:16.413296 22635 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.194:43739
I20240501 14:10:16.413341 22762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.194:43739 every 8 connection(s)
I20240501 14:10:16.414208 22635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 14:10:16.415493 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 22635
I20240501 14:10:16.415946 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:43545
--local_ip_for_outbound_sockets=127.6.153.195
--tserver_master_addrs=127.6.153.254:35355
--webserver_port=44769
--webserver_interface=127.6.153.195
--builtin_ntp_servers=127.6.153.212:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:16.423137 22763 heartbeater.cc:347] Connected to a master server at 127.6.153.254:35355
I20240501 14:10:16.423395 22763 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:16.423964 22763 heartbeater.cc:511] Master 127.6.153.254:35355 requested a full tablet report, sending...
I20240501 14:10:16.425146 21862 ts_manager.cc:194] Re-registered known tserver with Master: 84b20df3357443f5b174d9beca07daad (127.6.153.194:43739)
I20240501 14:10:16.425873 21862 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.194:55885
W20240501 14:10:16.515506 22766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:16.515703 22766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:16.515753 22766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:16.518601 22766 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:16.518657 22766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:16.518750 22766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:10:16.522218 22766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:43545
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=44769
--tserver_master_addrs=127.6.153.254:35355
--never_fsync=true
--heap_profile_path=/tmp/kudu.22766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:16.522967 22766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:16.524020 22766 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:16.531549 22772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:16.531782 22771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:16.532138 22774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:16.532550 22766 server_base.cc:1034] running on GCE node
I20240501 14:10:16.532917 22766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:16.533390 22766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:16.534535 22766 hybrid_clock.cc:648] HybridClock initialized: now 1714572616534522 us; error 26 us; skew 500 ppm
I20240501 14:10:16.536208 22766 webserver.cc:460] Webserver started at http://127.6.153.195:44769/ using document root <none> and password file <none>
I20240501 14:10:16.536708 22766 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:16.536778 22766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:16.539842 22766 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:10:16.541589 22780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:16.542510 22766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:16.542598 22766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "6ce85ae069f94132aa86b0879ad1607b"
format_stamp: "Formatted at 2024-05-01 14:10:10 on dist-test-slave-cmf0"
I20240501 14:10:16.542963 22766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:16.554962 22766 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:16.555444 22766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:16.555577 22766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:16.556017 22766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:16.556784 22766 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:16.556828 22766 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:16.556867 22766 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:16.556895 22766 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:16.564725 22766 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:43545
I20240501 14:10:16.564901 22893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:43545 every 8 connection(s)
I20240501 14:10:16.565635 22766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:10:16.570922 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 22766
I20240501 14:10:16.573483 22894 heartbeater.cc:347] Connected to a master server at 127.6.153.254:35355
I20240501 14:10:16.573665 22894 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:16.574107 22894 heartbeater.cc:511] Master 127.6.153.254:35355 requested a full tablet report, sending...
I20240501 14:10:16.575059 21862 ts_manager.cc:194] Re-registered known tserver with Master: 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195:43545)
I20240501 14:10:16.575608 21862 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:47165
I20240501 14:10:16.748159 22384 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:16.749348 21987 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:16.750499 22828 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:16.750838 22697 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:16.831540 21987 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:16.832182 22384 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:16.834583 22828 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:16.834693 22697 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:16.940946 22828 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:16.941622 22697 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:16.941679 21987 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:16.949717 22384 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:17.019731 22828 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:17.020530 21987 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 14:10:17.021137 22697 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:17.022163 22384 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:17.054323 22008 consensus_queue.cc:237] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.055611 22404 raft_consensus.cc:1261] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:17.055977 23036 consensus_queue.cc:1035] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:17.057766 22580 raft_consensus.cc:2932] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } } }
I20240501 14:10:17.058153 22404 raft_consensus.cc:2932] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } } }
W20240501 14:10:17.059906 21944 consensus_peers.cc:485] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 -> Peer 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195:43545): Couldn't send request to peer 6ce85ae069f94132aa86b0879ad1607b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d63f4896380141cd91e024c3ca83df35. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:17.060130 21862 catalog_manager.cc:5484] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:17.067392 22008 consensus_queue.cc:237] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.068797 22404 raft_consensus.cc:1261] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:17.069110 23036 consensus_queue.cc:1035] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:17.070526 22580 raft_consensus.cc:2932] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } } }
I20240501 14:10:17.070948 22403 raft_consensus.cc:2932] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } } }
W20240501 14:10:17.072124 21942 consensus_peers.cc:485] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 -> Peer 84b20df3357443f5b174d9beca07daad (127.6.153.194:43739): Couldn't send request to peer 84b20df3357443f5b174d9beca07daad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 30328a0a24b543ceb69cb752fb23726a. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:17.072587 21865 catalog_manager.cc:5484] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:17.078904 22403 consensus_queue.cc:237] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.080034 22008 raft_consensus.cc:1261] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:17.080351 23054 consensus_queue.cc:1035] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:17.082108 22633 raft_consensus.cc:2932] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } } }
I20240501 14:10:17.083168 22008 raft_consensus.cc:2932] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } } }
W20240501 14:10:17.083571 22340 consensus_peers.cc:485] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a -> Peer 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195:43545): Couldn't send request to peer 6ce85ae069f94132aa86b0879ad1607b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6d7ac45db904e25a7c57c781a5045cd. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:17.084784 21862 catalog_manager.cc:5484] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:17.090791 22008 consensus_queue.cc:237] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.091746 22403 raft_consensus.cc:1261] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:10:17.092017 21942 consensus_peers.cc:485] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 -> Peer 84b20df3357443f5b174d9beca07daad (127.6.153.194:43739): Couldn't send request to peer 84b20df3357443f5b174d9beca07daad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f6efade2ad54680956ecd013cd4a302. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:17.092067 22580 consensus_queue.cc:1035] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:17.093400 23036 raft_consensus.cc:2932] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } } }
I20240501 14:10:17.099402 22403 raft_consensus.cc:2932] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } } }
I20240501 14:10:17.100071 21865 catalog_manager.cc:5484] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:17.104727 22403 consensus_queue.cc:237] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.105813 22008 raft_consensus.cc:1261] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:10:17.105983 22340 consensus_peers.cc:485] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a -> Peer 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195:43545): Couldn't send request to peer 6ce85ae069f94132aa86b0879ad1607b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 083a3f99cc90422c953adb4be0afbc99. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:17.106113 23057 consensus_queue.cc:1035] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:17.107376 23054 raft_consensus.cc:2932] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } } }
I20240501 14:10:17.107810 22008 raft_consensus.cc:2932] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } } }
I20240501 14:10:17.109333 21865 catalog_manager.cc:5484] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index -1 to 4, NON_VOTER 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) added. New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:17.115936 22008 consensus_queue.cc:237] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.116933 22403 raft_consensus.cc:1261] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:10:17.117177 21942 consensus_peers.cc:485] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 -> Peer 84b20df3357443f5b174d9beca07daad (127.6.153.194:43739): Couldn't send request to peer 84b20df3357443f5b174d9beca07daad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a46c98a1ed44fb68c558bb034e10dc8. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:17.117436 23036 consensus_queue.cc:1035] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:17.118844 22580 raft_consensus.cc:2932] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } } }
I20240501 14:10:17.119200 22403 raft_consensus.cc:2932] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } } }
I20240501 14:10:17.121222 21862 catalog_manager.cc:5484] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index -1 to 4, NON_VOTER 84b20df3357443f5b174d9beca07daad (127.6.153.194) added. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } } }
I20240501 14:10:17.427961 22763 heartbeater.cc:503] Master 127.6.153.254:35355 was elected leader, sending a full tablet report...
I20240501 14:10:17.480333 23137 ts_tablet_manager.cc:902] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: Initiating tablet copy from peer 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:17.480999 23137 tablet_copy_client.cc:308] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:46839
I20240501 14:10:17.482821 23143 ts_tablet_manager.cc:902] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: Initiating tablet copy from peer 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:17.483505 23143 tablet_copy_client.cc:308] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:41495
I20240501 14:10:17.484948 22029 tablet_copy_service.cc:140] P 2ddd674017424cf78901b63950999865: Received BeginTabletCopySession request for tablet 1f6efade2ad54680956ecd013cd4a302 from peer 84b20df3357443f5b174d9beca07daad ({username='slave'} at 127.6.153.194:39849)
I20240501 14:10:17.485086 22029 tablet_copy_service.cc:161] P 2ddd674017424cf78901b63950999865: Beginning new tablet copy session on tablet 1f6efade2ad54680956ecd013cd4a302 from peer 84b20df3357443f5b174d9beca07daad at {username='slave'} at 127.6.153.194:39849: session id = 84b20df3357443f5b174d9beca07daad-1f6efade2ad54680956ecd013cd4a302
I20240501 14:10:17.486191 22029 tablet_copy_source_session.cc:205] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:17.487124 22425 tablet_copy_service.cc:140] P 5cdba816534e47eca8624ff41c9e6d9a: Received BeginTabletCopySession request for tablet 083a3f99cc90422c953adb4be0afbc99 from peer 6ce85ae069f94132aa86b0879ad1607b ({username='slave'} at 127.6.153.195:60559)
I20240501 14:10:17.487252 22425 tablet_copy_service.cc:161] P 5cdba816534e47eca8624ff41c9e6d9a: Beginning new tablet copy session on tablet 083a3f99cc90422c953adb4be0afbc99 from peer 6ce85ae069f94132aa86b0879ad1607b at {username='slave'} at 127.6.153.195:60559: session id = 6ce85ae069f94132aa86b0879ad1607b-083a3f99cc90422c953adb4be0afbc99
I20240501 14:10:17.488517 22425 tablet_copy_source_session.cc:205] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:17.489693 23137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6efade2ad54680956ecd013cd4a302. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:17.491137 23143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 083a3f99cc90422c953adb4be0afbc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:17.494554 23137 tablet_copy_client.cc:783] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:17.494714 23137 tablet_copy_client.cc:647] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:17.495887 23143 tablet_copy_client.cc:783] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:17.496158 23143 tablet_copy_client.cc:647] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:17.496317 23137 tablet_copy_client.cc:523] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:17.497572 23143 tablet_copy_client.cc:523] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:17.497725 23137 tablet_bootstrap.cc:492] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: Bootstrap starting.
I20240501 14:10:17.498853 23143 tablet_bootstrap.cc:492] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap starting.
I20240501 14:10:17.502849 23137 log.cc:824] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:17.503089 23143 log.cc:824] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:17.520996 23143 tablet_bootstrap.cc:492] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:17.521265 23137 tablet_bootstrap.cc:492] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:17.521283 23143 tablet_bootstrap.cc:492] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap complete.
I20240501 14:10:17.521517 23137 tablet_bootstrap.cc:492] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: Bootstrap complete.
I20240501 14:10:17.521546 23143 ts_tablet_manager.cc:1367] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.004s
I20240501 14:10:17.521762 23137 ts_tablet_manager.cc:1367] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.000s
I20240501 14:10:17.523265 23143 raft_consensus.cc:348] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.523370 23137 raft_consensus.cc:348] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.523485 23143 raft_consensus.cc:729] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6ce85ae069f94132aa86b0879ad1607b, State: Initialized, Role: LEARNER
I20240501 14:10:17.523551 23137 raft_consensus.cc:729] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 84b20df3357443f5b174d9beca07daad, State: Initialized, Role: LEARNER
I20240501 14:10:17.523766 23143 consensus_queue.cc:260] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.523798 23137 consensus_queue.cc:260] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.524327 22894 heartbeater.cc:503] Master 127.6.153.254:35355 was elected leader, sending a full tablet report...
I20240501 14:10:17.524536 23143 ts_tablet_manager.cc:1398] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20240501 14:10:17.524538 23137 ts_tablet_manager.cc:1398] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:17.525192 22425 tablet_copy_service.cc:342] P 5cdba816534e47eca8624ff41c9e6d9a: Request end of tablet copy session 6ce85ae069f94132aa86b0879ad1607b-083a3f99cc90422c953adb4be0afbc99 received from {username='slave'} at 127.6.153.195:60559
I20240501 14:10:17.525281 22425 tablet_copy_service.cc:433] P 5cdba816534e47eca8624ff41c9e6d9a: ending tablet copy session 6ce85ae069f94132aa86b0879ad1607b-083a3f99cc90422c953adb4be0afbc99 on tablet 083a3f99cc90422c953adb4be0afbc99 with peer 6ce85ae069f94132aa86b0879ad1607b
I20240501 14:10:17.526489 22029 tablet_copy_service.cc:342] P 2ddd674017424cf78901b63950999865: Request end of tablet copy session 84b20df3357443f5b174d9beca07daad-1f6efade2ad54680956ecd013cd4a302 received from {username='slave'} at 127.6.153.194:39849
I20240501 14:10:17.526585 22029 tablet_copy_service.cc:433] P 2ddd674017424cf78901b63950999865: ending tablet copy session 84b20df3357443f5b174d9beca07daad-1f6efade2ad54680956ecd013cd4a302 on tablet 1f6efade2ad54680956ecd013cd4a302 with peer 84b20df3357443f5b174d9beca07daad
I20240501 14:10:17.567847 23143 ts_tablet_manager.cc:902] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: Initiating tablet copy from peer 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:17.568272 23143 tablet_copy_client.cc:308] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:46839
I20240501 14:10:17.571563 22029 tablet_copy_service.cc:140] P 2ddd674017424cf78901b63950999865: Received BeginTabletCopySession request for tablet d63f4896380141cd91e024c3ca83df35 from peer 6ce85ae069f94132aa86b0879ad1607b ({username='slave'} at 127.6.153.195:43565)
I20240501 14:10:17.571664 22029 tablet_copy_service.cc:161] P 2ddd674017424cf78901b63950999865: Beginning new tablet copy session on tablet d63f4896380141cd91e024c3ca83df35 from peer 6ce85ae069f94132aa86b0879ad1607b at {username='slave'} at 127.6.153.195:43565: session id = 6ce85ae069f94132aa86b0879ad1607b-d63f4896380141cd91e024c3ca83df35
I20240501 14:10:17.572381 22029 tablet_copy_source_session.cc:205] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:17.573149 23143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63f4896380141cd91e024c3ca83df35. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:17.575126 23143 tablet_copy_client.cc:783] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:17.575263 23143 tablet_copy_client.cc:647] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:17.576202 23143 tablet_copy_client.cc:523] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:17.577134 23143 tablet_bootstrap.cc:492] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap starting.
I20240501 14:10:17.597374 23143 tablet_bootstrap.cc:492] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:17.597584 23143 tablet_bootstrap.cc:492] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap complete.
I20240501 14:10:17.597723 23143 ts_tablet_manager.cc:1367] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.000s
I20240501 14:10:17.598237 23143 raft_consensus.cc:348] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.598358 23143 raft_consensus.cc:729] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6ce85ae069f94132aa86b0879ad1607b, State: Initialized, Role: LEARNER
I20240501 14:10:17.598510 23143 consensus_queue.cc:260] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.598773 23143 ts_tablet_manager.cc:1398] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 14:10:17.599210 22029 tablet_copy_service.cc:342] P 2ddd674017424cf78901b63950999865: Request end of tablet copy session 6ce85ae069f94132aa86b0879ad1607b-d63f4896380141cd91e024c3ca83df35 received from {username='slave'} at 127.6.153.195:43565
I20240501 14:10:17.599282 22029 tablet_copy_service.cc:433] P 2ddd674017424cf78901b63950999865: ending tablet copy session 6ce85ae069f94132aa86b0879ad1607b-d63f4896380141cd91e024c3ca83df35 on tablet d63f4896380141cd91e024c3ca83df35 with peer 6ce85ae069f94132aa86b0879ad1607b
I20240501 14:10:17.607304 23137 ts_tablet_manager.cc:902] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: Initiating tablet copy from peer 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:17.607746 23137 tablet_copy_client.cc:308] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:46839
I20240501 14:10:17.608151 22029 tablet_copy_service.cc:140] P 2ddd674017424cf78901b63950999865: Received BeginTabletCopySession request for tablet 6a46c98a1ed44fb68c558bb034e10dc8 from peer 84b20df3357443f5b174d9beca07daad ({username='slave'} at 127.6.153.194:39849)
I20240501 14:10:17.608242 22029 tablet_copy_service.cc:161] P 2ddd674017424cf78901b63950999865: Beginning new tablet copy session on tablet 6a46c98a1ed44fb68c558bb034e10dc8 from peer 84b20df3357443f5b174d9beca07daad at {username='slave'} at 127.6.153.194:39849: session id = 84b20df3357443f5b174d9beca07daad-6a46c98a1ed44fb68c558bb034e10dc8
I20240501 14:10:17.608950 22029 tablet_copy_source_session.cc:205] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:17.609840 23192 ts_tablet_manager.cc:902] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: Initiating tablet copy from peer 2ddd674017424cf78901b63950999865 (127.6.153.193:46839)
I20240501 14:10:17.610296 23192 tablet_copy_client.cc:308] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:46839
I20240501 14:10:17.611737 22029 tablet_copy_service.cc:140] P 2ddd674017424cf78901b63950999865: Received BeginTabletCopySession request for tablet 30328a0a24b543ceb69cb752fb23726a from peer 84b20df3357443f5b174d9beca07daad ({username='slave'} at 127.6.153.194:39849)
I20240501 14:10:17.611824 22029 tablet_copy_service.cc:161] P 2ddd674017424cf78901b63950999865: Beginning new tablet copy session on tablet 30328a0a24b543ceb69cb752fb23726a from peer 84b20df3357443f5b174d9beca07daad at {username='slave'} at 127.6.153.194:39849: session id = 84b20df3357443f5b174d9beca07daad-30328a0a24b543ceb69cb752fb23726a
I20240501 14:10:17.611994 23137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a46c98a1ed44fb68c558bb034e10dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:17.612537 22029 tablet_copy_source_session.cc:205] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:17.613258 23192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30328a0a24b543ceb69cb752fb23726a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:17.613987 23137 tablet_copy_client.cc:783] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:17.614126 23137 tablet_copy_client.cc:647] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:17.615185 23192 tablet_copy_client.cc:783] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:17.615307 23192 tablet_copy_client.cc:647] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:17.616225 23192 tablet_copy_client.cc:523] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:17.617239 23192 tablet_bootstrap.cc:492] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: Bootstrap starting.
I20240501 14:10:17.620226 23137 tablet_copy_client.cc:523] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:17.621069 23137 tablet_bootstrap.cc:492] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: Bootstrap starting.
I20240501 14:10:17.636059 23192 tablet_bootstrap.cc:492] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:17.636299 23192 tablet_bootstrap.cc:492] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: Bootstrap complete.
I20240501 14:10:17.636448 23192 ts_tablet_manager.cc:1367] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20240501 14:10:17.636714 23143 ts_tablet_manager.cc:902] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: Initiating tablet copy from peer 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495)
I20240501 14:10:17.636926 23192 raft_consensus.cc:348] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.637106 23192 raft_consensus.cc:729] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 84b20df3357443f5b174d9beca07daad, State: Initialized, Role: LEARNER
I20240501 14:10:17.637185 23143 tablet_copy_client.cc:308] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:41495
I20240501 14:10:17.637261 23192 consensus_queue.cc:260] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.637583 23192 ts_tablet_manager.cc:1398] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:10:17.638022 23137 tablet_bootstrap.cc:492] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:17.638013 22425 tablet_copy_service.cc:140] P 5cdba816534e47eca8624ff41c9e6d9a: Received BeginTabletCopySession request for tablet a6d7ac45db904e25a7c57c781a5045cd from peer 6ce85ae069f94132aa86b0879ad1607b ({username='slave'} at 127.6.153.195:60559)
I20240501 14:10:17.638113 22425 tablet_copy_service.cc:161] P 5cdba816534e47eca8624ff41c9e6d9a: Beginning new tablet copy session on tablet a6d7ac45db904e25a7c57c781a5045cd from peer 6ce85ae069f94132aa86b0879ad1607b at {username='slave'} at 127.6.153.195:60559: session id = 6ce85ae069f94132aa86b0879ad1607b-a6d7ac45db904e25a7c57c781a5045cd
I20240501 14:10:17.638232 23137 tablet_bootstrap.cc:492] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: Bootstrap complete.
I20240501 14:10:17.638375 23137 ts_tablet_manager.cc:1367] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20240501 14:10:17.638715 23137 raft_consensus.cc:348] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.638810 23137 raft_consensus.cc:729] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 84b20df3357443f5b174d9beca07daad, State: Initialized, Role: LEARNER
I20240501 14:10:17.638891 23137 consensus_queue.cc:260] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: true } }
I20240501 14:10:17.638988 22425 tablet_copy_source_session.cc:205] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:17.639169 23137 ts_tablet_manager.cc:1398] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:10:17.639359 22029 tablet_copy_service.cc:342] P 2ddd674017424cf78901b63950999865: Request end of tablet copy session 84b20df3357443f5b174d9beca07daad-30328a0a24b543ceb69cb752fb23726a received from {username='slave'} at 127.6.153.194:39849
I20240501 14:10:17.639425 22029 tablet_copy_service.cc:433] P 2ddd674017424cf78901b63950999865: ending tablet copy session 84b20df3357443f5b174d9beca07daad-30328a0a24b543ceb69cb752fb23726a on tablet 30328a0a24b543ceb69cb752fb23726a with peer 84b20df3357443f5b174d9beca07daad
I20240501 14:10:17.639806 23143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d7ac45db904e25a7c57c781a5045cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:17.641502 23143 tablet_copy_client.cc:783] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:17.641651 23143 tablet_copy_client.cc:647] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:17.641799 22028 tablet_copy_service.cc:342] P 2ddd674017424cf78901b63950999865: Request end of tablet copy session 84b20df3357443f5b174d9beca07daad-6a46c98a1ed44fb68c558bb034e10dc8 received from {username='slave'} at 127.6.153.194:39849
I20240501 14:10:17.641856 22028 tablet_copy_service.cc:433] P 2ddd674017424cf78901b63950999865: ending tablet copy session 84b20df3357443f5b174d9beca07daad-6a46c98a1ed44fb68c558bb034e10dc8 on tablet 6a46c98a1ed44fb68c558bb034e10dc8 with peer 84b20df3357443f5b174d9beca07daad
I20240501 14:10:17.642637 23143 tablet_copy_client.cc:523] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:17.643535 23143 tablet_bootstrap.cc:492] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap starting.
I20240501 14:10:17.663811 23143 tablet_bootstrap.cc:492] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:17.664012 23143 tablet_bootstrap.cc:492] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: Bootstrap complete.
I20240501 14:10:17.664142 23143 ts_tablet_manager.cc:1367] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.000s
I20240501 14:10:17.664482 23143 raft_consensus.cc:348] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.664575 23143 raft_consensus.cc:729] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6ce85ae069f94132aa86b0879ad1607b, State: Initialized, Role: LEARNER
I20240501 14:10:17.664682 23143 consensus_queue.cc:260] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: true } }
I20240501 14:10:17.664888 23143 ts_tablet_manager.cc:1398] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:17.665369 22425 tablet_copy_service.cc:342] P 5cdba816534e47eca8624ff41c9e6d9a: Request end of tablet copy session 6ce85ae069f94132aa86b0879ad1607b-a6d7ac45db904e25a7c57c781a5045cd received from {username='slave'} at 127.6.153.195:60559
I20240501 14:10:17.665436 22425 tablet_copy_service.cc:433] P 5cdba816534e47eca8624ff41c9e6d9a: ending tablet copy session 6ce85ae069f94132aa86b0879ad1607b-a6d7ac45db904e25a7c57c781a5045cd on tablet a6d7ac45db904e25a7c57c781a5045cd with peer 6ce85ae069f94132aa86b0879ad1607b
I20240501 14:10:17.958539 22717 raft_consensus.cc:1203] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:10:18.011626 22717 raft_consensus.cc:1203] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:10:18.040602 22848 raft_consensus.cc:1203] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:10:18.109405 22848 raft_consensus.cc:1203] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:10:18.216059 22717 raft_consensus.cc:1203] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:10:18.236168 22848 raft_consensus.cc:1203] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 14:10:18.373881 23234 raft_consensus.cc:1050] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865: attempting to promote NON_VOTER 84b20df3357443f5b174d9beca07daad to VOTER
I20240501 14:10:18.374317 23234 consensus_queue.cc:237] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } }
I20240501 14:10:18.375522 22403 raft_consensus.cc:1261] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.375557 22717 raft_consensus.cc:1261] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.375970 23235 consensus_queue.cc:1035] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.376163 23036 consensus_queue.cc:1035] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.377791 23036 raft_consensus.cc:2932] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.378324 22403 raft_consensus.cc:2932] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.379200 22717 raft_consensus.cc:2932] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.380244 21862 catalog_manager.cc:5484] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:18.383738 22008 consensus_queue.cc:237] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } }
I20240501 14:10:18.384856 22717 raft_consensus.cc:1261] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:18.385141 23235 consensus_queue.cc:1035] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:18.386385 23234 raft_consensus.cc:2932] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.386807 22717 raft_consensus.cc:2932] T 1f6efade2ad54680956ecd013cd4a302 P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.387616 21854 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1f6efade2ad54680956ecd013cd4a302 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:18.388401 21865 catalog_manager.cc:5484] T 1f6efade2ad54680956ecd013cd4a302 P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:18.390321 22384 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f6efade2ad54680956ecd013cd4a302 with delete_type TABLET_DATA_TOMBSTONED (TS 5cdba816534e47eca8624ff41c9e6d9a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57890
I20240501 14:10:18.390728 23305 tablet_replica.cc:330] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: stopping tablet replica
I20240501 14:10:18.390950 23305 raft_consensus.cc:2229] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:18.391091 23305 raft_consensus.cc:2258] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:18.391958 23305 ts_tablet_manager.cc:1875] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:18.393546 23305 ts_tablet_manager.cc:1888] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:10:18.393615 23305 log.cc:1196] T 1f6efade2ad54680956ecd013cd4a302 P 5cdba816534e47eca8624ff41c9e6d9a: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/1f6efade2ad54680956ecd013cd4a302
I20240501 14:10:18.394026 21854 catalog_manager.cc:4830] TS 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495): tablet 1f6efade2ad54680956ecd013cd4a302 (table rebalance_test_table_1 [id=5bae9f77612c4fb09bc909d151ea4d09]) successfully deleted
I20240501 14:10:18.491433 23236 raft_consensus.cc:1050] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a: attempting to promote NON_VOTER 6ce85ae069f94132aa86b0879ad1607b to VOTER
I20240501 14:10:18.491946 23236 consensus_queue.cc:237] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:18.493188 22848 raft_consensus.cc:1261] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.493209 22008 raft_consensus.cc:1261] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.493589 23237 consensus_queue.cc:1035] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.493762 23057 consensus_queue.cc:1035] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.495409 23237 raft_consensus.cc:2932] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.495859 22008 raft_consensus.cc:2932] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.496807 22848 raft_consensus.cc:2932] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.498852 21862 catalog_manager.cc:5484] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:18.501924 22403 consensus_queue.cc:237] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:18.502847 22848 raft_consensus.cc:1261] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:18.502911 23235 raft_consensus.cc:1050] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865: attempting to promote NON_VOTER 84b20df3357443f5b174d9beca07daad to VOTER
I20240501 14:10:18.503196 23235 consensus_queue.cc:237] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } }
I20240501 14:10:18.503585 23236 consensus_queue.cc:1035] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:18.504454 22403 raft_consensus.cc:1261] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.504774 22717 raft_consensus.cc:1261] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.505167 23234 consensus_queue.cc:1035] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.505404 23236 raft_consensus.cc:2932] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.505470 23234 consensus_queue.cc:1035] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.506743 22848 raft_consensus.cc:2932] T 083a3f99cc90422c953adb4be0afbc99 P 6ce85ae069f94132aa86b0879ad1607b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.506970 21854 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 083a3f99cc90422c953adb4be0afbc99 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:18.507437 21861 catalog_manager.cc:5484] T 083a3f99cc90422c953adb4be0afbc99 P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index 5 to 6, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:18.507577 23234 raft_consensus.cc:2932] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.509665 22403 raft_consensus.cc:2932] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.510069 21987 tablet_service.cc:1514] Processing DeleteTablet for tablet 083a3f99cc90422c953adb4be0afbc99 with delete_type TABLET_DATA_TOMBSTONED (TS 2ddd674017424cf78901b63950999865 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39584
I20240501 14:10:18.510478 23321 tablet_replica.cc:330] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: stopping tablet replica
I20240501 14:10:18.510489 22717 raft_consensus.cc:2932] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.510799 23321 raft_consensus.cc:2229] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:18.510989 23321 raft_consensus.cc:2258] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:18.511047 21862 catalog_manager.cc:5484] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:18.512331 23321 ts_tablet_manager.cc:1875] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:18.514791 23321 ts_tablet_manager.cc:1888] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:10:18.514927 23321 log.cc:1196] T 083a3f99cc90422c953adb4be0afbc99 P 2ddd674017424cf78901b63950999865: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/083a3f99cc90422c953adb4be0afbc99
I20240501 14:10:18.515450 22008 consensus_queue.cc:237] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } }
I20240501 14:10:18.515554 21854 catalog_manager.cc:4830] TS 2ddd674017424cf78901b63950999865 (127.6.153.193:46839): tablet 083a3f99cc90422c953adb4be0afbc99 (table rebalance_test_table_2 [id=ed1ae7e241284581a321010797e0e9b4]) successfully deleted
I20240501 14:10:18.516265 22717 raft_consensus.cc:1261] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:18.516630 23235 consensus_queue.cc:1035] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:18.518008 23036 raft_consensus.cc:2932] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.519533 21854 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 30328a0a24b543ceb69cb752fb23726a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:18.519659 22717 raft_consensus.cc:2932] T 30328a0a24b543ceb69cb752fb23726a P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.520046 21863 catalog_manager.cc:5484] T 30328a0a24b543ceb69cb752fb23726a P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:18.522248 22384 tablet_service.cc:1514] Processing DeleteTablet for tablet 30328a0a24b543ceb69cb752fb23726a with delete_type TABLET_DATA_TOMBSTONED (TS 5cdba816534e47eca8624ff41c9e6d9a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57890
I20240501 14:10:18.522429 23305 tablet_replica.cc:330] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: stopping tablet replica
I20240501 14:10:18.522553 23305 raft_consensus.cc:2229] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:18.522692 23305 raft_consensus.cc:2258] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:18.523244 23305 ts_tablet_manager.cc:1875] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:18.524794 23305 ts_tablet_manager.cc:1888] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:10:18.524855 23305 log.cc:1196] T 30328a0a24b543ceb69cb752fb23726a P 5cdba816534e47eca8624ff41c9e6d9a: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/30328a0a24b543ceb69cb752fb23726a
I20240501 14:10:18.525256 21854 catalog_manager.cc:4830] TS 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495): tablet 30328a0a24b543ceb69cb752fb23726a (table rebalance_test_table_2 [id=ed1ae7e241284581a321010797e0e9b4]) successfully deleted
I20240501 14:10:18.730324 23036 raft_consensus.cc:1050] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: attempting to promote NON_VOTER 6ce85ae069f94132aa86b0879ad1607b to VOTER
I20240501 14:10:18.730664 23036 consensus_queue.cc:237] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:18.731846 22848 raft_consensus.cc:1261] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.731901 22403 raft_consensus.cc:1261] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.732189 23234 consensus_queue.cc:1035] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.732381 23235 consensus_queue.cc:1035] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.733955 23036 raft_consensus.cc:2932] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.734351 22848 raft_consensus.cc:2932] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.734491 22403 raft_consensus.cc:2932] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.736582 21861 catalog_manager.cc:5484] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:18.771458 22008 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35"
dest_uuid: "2ddd674017424cf78901b63950999865"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38504
I20240501 14:10:18.771636 22008 raft_consensus.cc:595] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:18.775815 23235 raft_consensus.cc:1050] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865: attempting to promote NON_VOTER 84b20df3357443f5b174d9beca07daad to VOTER
I20240501 14:10:18.776189 23235 consensus_queue.cc:237] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } }
I20240501 14:10:18.777496 22403 raft_consensus.cc:1261] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.777750 22717 raft_consensus.cc:1261] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad [term 1 LEARNER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.777933 23036 consensus_queue.cc:1035] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.778645 23036 consensus_queue.cc:1035] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.779915 23235 raft_consensus.cc:2932] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.780259 22403 raft_consensus.cc:2932] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.782306 21863 catalog_manager.cc:5484] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.783460 22717 raft_consensus.cc:2932] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 84b20df3357443f5b174d9beca07daad (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } attrs { replace: true } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.787451 22008 consensus_queue.cc:237] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } }
I20240501 14:10:18.788236 22717 raft_consensus.cc:1261] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Refusing update from remote peer 2ddd674017424cf78901b63950999865: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:18.788614 23382 consensus_queue.cc:1035] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:18.788753 23057 raft_consensus.cc:1050] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a: attempting to promote NON_VOTER 6ce85ae069f94132aa86b0879ad1607b to VOTER
I20240501 14:10:18.789088 23057 consensus_queue.cc:237] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:18.790356 22848 raft_consensus.cc:1261] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b [term 1 LEARNER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.790377 22008 raft_consensus.cc:1261] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:18.790689 23237 consensus_queue.cc:1035] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.790838 23235 raft_consensus.cc:2932] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.791103 23237 consensus_queue.cc:1035] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:18.791734 22717 raft_consensus.cc:2932] T 6a46c98a1ed44fb68c558bb034e10dc8 P 84b20df3357443f5b174d9beca07daad [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } } }
I20240501 14:10:18.792129 21854 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6a46c98a1ed44fb68c558bb034e10dc8 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:18.792836 21861 catalog_manager.cc:5484] T 6a46c98a1ed44fb68c558bb034e10dc8 P 2ddd674017424cf78901b63950999865 reported cstate change: config changed from index 5 to 6, VOTER 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "2ddd674017424cf78901b63950999865" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b20df3357443f5b174d9beca07daad" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 43739 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:18.794191 23237 raft_consensus.cc:2932] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.795749 22008 raft_consensus.cc:2932] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.795977 21863 catalog_manager.cc:5484] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:18.797083 22384 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a46c98a1ed44fb68c558bb034e10dc8 with delete_type TABLET_DATA_TOMBSTONED (TS 5cdba816534e47eca8624ff41c9e6d9a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57890
I20240501 14:10:18.797250 23305 tablet_replica.cc:330] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: stopping tablet replica
I20240501 14:10:18.797356 23305 raft_consensus.cc:2229] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:18.797483 23305 raft_consensus.cc:2258] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:18.798190 23305 ts_tablet_manager.cc:1875] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:18.799701 23305 ts_tablet_manager.cc:1888] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:10:18.799764 23305 log.cc:1196] T 6a46c98a1ed44fb68c558bb034e10dc8 P 5cdba816534e47eca8624ff41c9e6d9a: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/6a46c98a1ed44fb68c558bb034e10dc8
I20240501 14:10:18.800282 21854 catalog_manager.cc:4830] TS 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196:41495): tablet 6a46c98a1ed44fb68c558bb034e10dc8 (table rebalance_test_table_0 [id=c6701d34190b4ec3bd5e2ac772a65087]) successfully deleted
I20240501 14:10:18.804405 22848 raft_consensus.cc:2932] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.805971 22403 consensus_queue.cc:237] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:18.806929 22848 raft_consensus.cc:1261] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b [term 1 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:18.807252 23237 consensus_queue.cc:1035] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:18.808568 23236 raft_consensus.cc:2932] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.809016 22848 raft_consensus.cc:2932] T a6d7ac45db904e25a7c57c781a5045cd P 6ce85ae069f94132aa86b0879ad1607b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:18.809599 21854 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a6d7ac45db904e25a7c57c781a5045cd with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:18.810374 21863 catalog_manager.cc:5484] T a6d7ac45db904e25a7c57c781a5045cd P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index 5 to 6, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:18.812431 21987 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d7ac45db904e25a7c57c781a5045cd with delete_type TABLET_DATA_TOMBSTONED (TS 2ddd674017424cf78901b63950999865 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39584
I20240501 14:10:18.812613 23321 tablet_replica.cc:330] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: stopping tablet replica
I20240501 14:10:18.812733 23321 raft_consensus.cc:2229] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:18.812853 23321 raft_consensus.cc:2258] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:18.813469 23321 ts_tablet_manager.cc:1875] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:18.815448 23321 ts_tablet_manager.cc:1888] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 14:10:18.815521 23321 log.cc:1196] T a6d7ac45db904e25a7c57c781a5045cd P 2ddd674017424cf78901b63950999865: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/a6d7ac45db904e25a7c57c781a5045cd
I20240501 14:10:18.815970 21854 catalog_manager.cc:4830] TS 2ddd674017424cf78901b63950999865 (127.6.153.193:46839): tablet a6d7ac45db904e25a7c57c781a5045cd (table rebalance_test_table_1 [id=5bae9f77612c4fb09bc909d151ea4d09]) successfully deleted
I20240501 14:10:18.992583 22008 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35"
dest_uuid: "2ddd674017424cf78901b63950999865"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38532
I20240501 14:10:18.992702 22008 raft_consensus.cc:595] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:19.005570 22008 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35"
dest_uuid: "2ddd674017424cf78901b63950999865"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38554
I20240501 14:10:19.005688 22008 raft_consensus.cc:595] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:19.214937 22008 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35"
dest_uuid: "2ddd674017424cf78901b63950999865"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38580
I20240501 14:10:19.215063 22008 raft_consensus.cc:595] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:19.252244 23234 raft_consensus.cc:979] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: : Instructing follower 5cdba816534e47eca8624ff41c9e6d9a to start an election
I20240501 14:10:19.252372 23234 raft_consensus.cc:1067] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Signalling peer 5cdba816534e47eca8624ff41c9e6d9a to start an election
I20240501 14:10:19.252904 22403 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35"
dest_uuid: "5cdba816534e47eca8624ff41c9e6d9a"
from {username='slave'} at 127.6.153.193:39499
I20240501 14:10:19.253051 22403 raft_consensus.cc:482] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:10:19.253099 22403 raft_consensus.cc:3037] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:19.253877 22403 raft_consensus.cc:504] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:19.254241 22403 leader_election.cc:288] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 2 election: Requested vote from peers 2ddd674017424cf78901b63950999865 (127.6.153.193:46839), 6ce85ae069f94132aa86b0879ad1607b (127.6.153.195:43545)
I20240501 14:10:19.254518 22008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35" candidate_uuid: "5cdba816534e47eca8624ff41c9e6d9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "2ddd674017424cf78901b63950999865"
I20240501 14:10:19.254650 22008 raft_consensus.cc:3032] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:10:19.254700 22008 raft_consensus.cc:729] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ddd674017424cf78901b63950999865, State: Running, Role: LEADER
I20240501 14:10:19.254698 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63f4896380141cd91e024c3ca83df35" candidate_uuid: "5cdba816534e47eca8624ff41c9e6d9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "6ce85ae069f94132aa86b0879ad1607b"
I20240501 14:10:19.254874 22848 raft_consensus.cc:3037] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:19.254848 22008 consensus_queue.cc:260] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:19.255013 22008 raft_consensus.cc:3037] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:19.255656 22848 raft_consensus.cc:2454] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdba816534e47eca8624ff41c9e6d9a in term 2.
I20240501 14:10:19.255766 22008 raft_consensus.cc:2454] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdba816534e47eca8624ff41c9e6d9a in term 2.
I20240501 14:10:19.255998 22340 leader_election.cc:302] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cdba816534e47eca8624ff41c9e6d9a, 6ce85ae069f94132aa86b0879ad1607b; no voters:
I20240501 14:10:19.256151 23237 raft_consensus.cc:2781] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:10:19.256235 23237 raft_consensus.cc:686] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 2 LEADER]: Becoming Leader. State: Replica: 5cdba816534e47eca8624ff41c9e6d9a, State: Running, Role: LEADER
I20240501 14:10:19.256363 23237 consensus_queue.cc:237] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:19.257776 21862 catalog_manager.cc:5484] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: term changed from 1 to 2, leader changed from 2ddd674017424cf78901b63950999865 (127.6.153.193) to 5cdba816534e47eca8624ff41c9e6d9a (127.6.153.196). New cstate: current_term: 2 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:19.645601 22848 raft_consensus.cc:1261] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 2 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 14:10:19.646091 23237 consensus_queue.cc:1035] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:19.650250 22008 raft_consensus.cc:1261] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 2 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 14:10:19.650796 23236 consensus_queue.cc:1035] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ddd674017424cf78901b63950999865" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 46839 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:19.652292 22403 consensus_queue.cc:237] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } }
I20240501 14:10:19.653153 22848 raft_consensus.cc:1261] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 2 FOLLOWER]: Refusing update from remote peer 5cdba816534e47eca8624ff41c9e6d9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 14:10:19.653494 23236 consensus_queue.cc:1035] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:19.654850 23237 raft_consensus.cc:2932] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:19.655375 22848 raft_consensus.cc:2932] T d63f4896380141cd91e024c3ca83df35 P 6ce85ae069f94132aa86b0879ad1607b [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } } }
I20240501 14:10:19.656102 21854 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d63f4896380141cd91e024c3ca83df35 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:19.657225 21862 catalog_manager.cc:5484] T d63f4896380141cd91e024c3ca83df35 P 5cdba816534e47eca8624ff41c9e6d9a reported cstate change: config changed from index 5 to 7, VOTER 2ddd674017424cf78901b63950999865 (127.6.153.193) evicted. New cstate: current_term: 2 leader_uuid: "5cdba816534e47eca8624ff41c9e6d9a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5cdba816534e47eca8624ff41c9e6d9a" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 41495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce85ae069f94132aa86b0879ad1607b" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 43545 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:19.659235 21987 tablet_service.cc:1514] Processing DeleteTablet for tablet d63f4896380141cd91e024c3ca83df35 with delete_type TABLET_DATA_TOMBSTONED (TS 2ddd674017424cf78901b63950999865 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:39584
I20240501 14:10:19.659551 23442 tablet_replica.cc:330] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: stopping tablet replica
I20240501 14:10:19.659679 23442 raft_consensus.cc:2229] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:19.659776 23442 raft_consensus.cc:2258] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:19.660254 23442 ts_tablet_manager.cc:1875] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:19.661928 23442 ts_tablet_manager.cc:1888] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 14:10:19.661993 23442 log.cc:1196] T d63f4896380141cd91e024c3ca83df35 P 2ddd674017424cf78901b63950999865: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/d63f4896380141cd91e024c3ca83df35
I20240501 14:10:19.662433 21854 catalog_manager.cc:4830] TS 2ddd674017424cf78901b63950999865 (127.6.153.193:46839): tablet d63f4896380141cd91e024c3ca83df35 (table rebalance_test_table_0 [id=c6701d34190b4ec3bd5e2ac772a65087]) successfully deleted
I20240501 14:10:19.877986 22828 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:19.879105 22384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:19.879495 21987 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:19.887882 22697 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:19.957274 22828 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:19.957752 22384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:19.958029 21987 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:19.959306 22697 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:20.034179 22384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:20.035368 21987 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:20.035739 22697 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:20.036932 22828 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:20.070935 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 21927
I20240501 14:10:20.076871 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 22635
I20240501 14:10:20.081667 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 22766
I20240501 14:10:20.086272 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 22322
I20240501 14:10:20.090997 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 21836
2024-05-01T14:10:20Z chronyd exiting
[ OK ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1 (10373 ms)
[ RUN ] LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer/idx6_rf7_t7_l_3_4_3
I20240501 14:10:20.113782 6759 test_util.cc:268] Using random seed: 200714096
I20240501 14:10:20.114526 6759 ts_itest-base.cc:112] Starting cluster with:
I20240501 14:10:20.114564 6759 ts_itest-base.cc:113] --------------
I20240501 14:10:20.114573 6759 ts_itest-base.cc:114] 10 tablet servers
I20240501 14:10:20.114583 6759 ts_itest-base.cc:115] 7 replicas per TS
I20240501 14:10:20.114591 6759 ts_itest-base.cc:116] --------------
2024-05-01T14:10:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T14:10:20Z Disabled control of system clock
I20240501 14:10:20.121942 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:34031
--webserver_interface=127.6.153.254
--webserver_port=0
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.153.254:34031
--location_mapping_cmd=/tmp/dist-test-taskxMccU0/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240501 14:10:20.237532 23559 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 14:10:20.237744 23559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:20.237785 23559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:20.237843 23559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:20.240732 23559 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:20.240797 23559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 14:10:20.240834 23559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:20.240864 23559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 14:10:20.240887 23559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 14:10:20.244357 23559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskxMccU0/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--ipki_ca_key_size=768
--master_addresses=127.6.153.254:34031
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.153.254:34031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.6.153.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.23559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:20.245095 23559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:20.246089 23559 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:20.252599 23567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:20.253013 23565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:20.253403 23564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:20.253796 23559 server_base.cc:1034] running on GCE node
I20240501 14:10:20.254236 23559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:20.255015 23559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:20.256161 23559 hybrid_clock.cc:648] HybridClock initialized: now 1714572620256150 us; error 22 us; skew 500 ppm
I20240501 14:10:20.257802 23559 webserver.cc:460] Webserver started at http://127.6.153.254:41101/ using document root <none> and password file <none>
I20240501 14:10:20.258298 23559 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:20.258352 23559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:20.258500 23559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:20.260007 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "10abebbc5591447aa4f2e32aa2a425f4"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.260444 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "10abebbc5591447aa4f2e32aa2a425f4"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.263458 23559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:10:20.265388 23573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.266269 23559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:20.266377 23559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
uuid: "10abebbc5591447aa4f2e32aa2a425f4"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.266453 23559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:20.288547 23559 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:20.289160 23559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:20.289310 23559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:20.295225 23559 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.254:34031
I20240501 14:10:20.295233 23625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.254:34031 every 8 connection(s)
I20240501 14:10:20.296293 23559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 14:10:20.297410 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 23559
I20240501 14:10:20.297539 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 14:10:20.299185 23626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:20.304148 23626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4: Bootstrap starting.
I20240501 14:10:20.306211 23626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:20.306869 23626 log.cc:824] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:20.308244 23626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4: No bootstrap required, opened a new log
I20240501 14:10:20.310458 23626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10abebbc5591447aa4f2e32aa2a425f4" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 34031 } }
I20240501 14:10:20.310621 23626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:20.310664 23626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10abebbc5591447aa4f2e32aa2a425f4, State: Initialized, Role: FOLLOWER
I20240501 14:10:20.311008 23626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10abebbc5591447aa4f2e32aa2a425f4" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 34031 } }
I20240501 14:10:20.311120 23626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 14:10:20.311178 23626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 14:10:20.311281 23626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:20.312270 23626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10abebbc5591447aa4f2e32aa2a425f4" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 34031 } }
I20240501 14:10:20.312556 23626 leader_election.cc:302] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10abebbc5591447aa4f2e32aa2a425f4; no voters:
I20240501 14:10:20.312788 23626 leader_election.cc:288] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 14:10:20.312885 23631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:20.313074 23631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [term 1 LEADER]: Becoming Leader. State: Replica: 10abebbc5591447aa4f2e32aa2a425f4, State: Running, Role: LEADER
I20240501 14:10:20.313364 23631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10abebbc5591447aa4f2e32aa2a425f4" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 34031 } }
I20240501 14:10:20.313735 23626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 14:10:20.314754 23632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10abebbc5591447aa4f2e32aa2a425f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10abebbc5591447aa4f2e32aa2a425f4" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 34031 } } }
I20240501 14:10:20.314888 23632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [sys.catalog]: This master's current role is: LEADER
I20240501 14:10:20.315290 23633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10abebbc5591447aa4f2e32aa2a425f4. Latest consensus state: current_term: 1 leader_uuid: "10abebbc5591447aa4f2e32aa2a425f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10abebbc5591447aa4f2e32aa2a425f4" member_type: VOTER last_known_addr { host: "127.6.153.254" port: 34031 } } }
I20240501 14:10:20.315459 23633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4 [sys.catalog]: This master's current role is: LEADER
W20240501 14:10:20.317960 23646 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240501 14:10:20.318020 23646 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240501 14:10:20.318095 23640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 14:10:20.318955 23640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 14:10:20.323360 23640 catalog_manager.cc:1347] Generated new cluster ID: 02f6d1b171ca4c8d88a20320d735b691
I20240501 14:10:20.323419 23640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 14:10:20.337358 23640 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 14:10:20.338186 23640 catalog_manager.cc:1504] Loading token signing keys...
I20240501 14:10:20.345968 23640 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 10abebbc5591447aa4f2e32aa2a425f4: Generated new TSK 0
I20240501 14:10:20.346554 23640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 14:10:20.355521 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.193:0
--local_ip_for_outbound_sockets=127.6.153.193
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:10:20.455142 23650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:20.455312 23650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:20.455384 23650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:20.458261 23650 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:20.458314 23650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:20.458390 23650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.193
I20240501 14:10:20.461808 23650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.6.153.193
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.23650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.193
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:20.462507 23650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:20.463481 23650 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:20.470655 23656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:20.470979 23658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:20.471263 23655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:20.471709 23650 server_base.cc:1034] running on GCE node
I20240501 14:10:20.472002 23650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:20.472457 23650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:20.473596 23650 hybrid_clock.cc:648] HybridClock initialized: now 1714572620473583 us; error 27 us; skew 500 ppm
I20240501 14:10:20.475176 23650 webserver.cc:460] Webserver started at http://127.6.153.193:45601/ using document root <none> and password file <none>
I20240501 14:10:20.475677 23650 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:20.475733 23650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:20.475883 23650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:20.477291 23650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "24d90b55560e45feb5fe4b9482317a4c"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.477700 23650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "24d90b55560e45feb5fe4b9482317a4c"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.480612 23650 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:20.482409 23664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.483245 23650 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:20.483362 23650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "24d90b55560e45feb5fe4b9482317a4c"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.483434 23650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:20.520287 23650 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:20.520859 23650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:20.521003 23650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:20.521504 23650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:20.522328 23650 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:20.522373 23650 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.522423 23650 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:20.522449 23650 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.529755 23650 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.193:35621
I20240501 14:10:20.529786 23777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.193:35621 every 8 connection(s)
I20240501 14:10:20.530581 23650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 14:10:20.538193 23778 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:20.538368 23778 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:20.538796 23778 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:20.540627 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 23650
I20240501 14:10:20.540735 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 14:10:20.544210 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.194:0
--local_ip_for_outbound_sockets=127.6.153.194
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:20.582327 23590 ts_manager.cc:194] Registered new tserver with Master: 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:20.583609 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.193:40257
W20240501 14:10:20.654935 23782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:20.655105 23782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:20.655153 23782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:20.658085 23782 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:20.658147 23782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:20.658221 23782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.194
I20240501 14:10:20.661649 23782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.6.153.194
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.23782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.194
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:20.662372 23782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:20.663367 23782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:20.670738 23790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:20.671087 23787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:20.671404 23788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:20.671826 23782 server_base.cc:1034] running on GCE node
I20240501 14:10:20.672125 23782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:20.672585 23782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:20.673730 23782 hybrid_clock.cc:648] HybridClock initialized: now 1714572620673720 us; error 26 us; skew 500 ppm
I20240501 14:10:20.675356 23782 webserver.cc:460] Webserver started at http://127.6.153.194:44789/ using document root <none> and password file <none>
I20240501 14:10:20.675851 23782 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:20.675904 23782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:20.676059 23782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:20.677505 23782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "2502b9e8098b4bc5af7204835fce444b"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.677933 23782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "2502b9e8098b4bc5af7204835fce444b"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.680848 23782 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:20.682749 23796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.683684 23782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:20.683784 23782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "2502b9e8098b4bc5af7204835fce444b"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.683853 23782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:20.716511 23782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:20.717047 23782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:20.717190 23782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:20.717672 23782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:20.718468 23782 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:20.718514 23782 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.718564 23782 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:20.718596 23782 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.726137 23782 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.194:34427
I20240501 14:10:20.726183 23909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.194:34427 every 8 connection(s)
I20240501 14:10:20.726959 23782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 14:10:20.729431 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 23782
I20240501 14:10:20.729571 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 14:10:20.731834 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.195:0
--local_ip_for_outbound_sockets=127.6.153.195
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:20.735538 23910 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:20.735723 23910 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:20.736181 23910 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:20.770378 23590 ts_manager.cc:194] Registered new tserver with Master: 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427)
I20240501 14:10:20.771068 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.194:49579
W20240501 14:10:20.832450 23913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:20.832634 23913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:20.832690 23913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:20.835649 23913 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:20.835711 23913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:20.835788 23913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.195
I20240501 14:10:20.839485 23913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.6.153.195
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.23913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.195
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:20.840250 23913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:20.841266 23913 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:20.848678 23920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:20.849028 23919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:20.849413 23922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:20.849808 23913 server_base.cc:1034] running on GCE node
I20240501 14:10:20.850121 23913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:20.850582 23913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:20.851734 23913 hybrid_clock.cc:648] HybridClock initialized: now 1714572620851710 us; error 37 us; skew 500 ppm
I20240501 14:10:20.853379 23913 webserver.cc:460] Webserver started at http://127.6.153.195:40779/ using document root <none> and password file <none>
I20240501 14:10:20.853875 23913 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:20.853996 23913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:20.854202 23913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:20.855713 23913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.856146 23913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.859037 23913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 14:10:20.860950 23928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.861831 23913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:20.861927 23913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
format_stamp: "Formatted at 2024-05-01 14:10:20 on dist-test-slave-cmf0"
I20240501 14:10:20.862008 23913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:20.880649 23913 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:20.881110 23913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:20.881242 23913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:20.881700 23913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:20.882481 23913 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:20.882527 23913 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.882575 23913 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:20.882604 23913 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:20.890277 23913 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.195:44313
I20240501 14:10:20.890300 24041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.195:44313 every 8 connection(s)
I20240501 14:10:20.891065 23913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 14:10:20.896817 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 23913
I20240501 14:10:20.896948 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 14:10:20.898983 24042 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:20.899016 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.196:0
--local_ip_for_outbound_sockets=127.6.153.196
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:20.899170 24042 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:20.899650 24042 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:20.942493 23590 ts_manager.cc:194] Registered new tserver with Master: 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:20.943212 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.195:37601
W20240501 14:10:21.011380 24045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:21.011547 24045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:21.011593 24045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:21.014513 24045 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:21.014564 24045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:21.014640 24045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.196
I20240501 14:10:21.018051 24045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.6.153.196
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.24045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.196
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:21.018743 24045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:21.019743 24045 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:21.026815 24054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.027194 24052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.027515 24051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:21.027920 24045 server_base.cc:1034] running on GCE node
I20240501 14:10:21.028249 24045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:21.028710 24045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:21.029855 24045 hybrid_clock.cc:648] HybridClock initialized: now 1714572621029835 us; error 27 us; skew 500 ppm
I20240501 14:10:21.031498 24045 webserver.cc:460] Webserver started at http://127.6.153.196:46855/ using document root <none> and password file <none>
I20240501 14:10:21.032008 24045 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:21.032060 24045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:21.032202 24045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:21.033629 24045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "166d7170d98944e88de2eccda9d30545"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.034034 24045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "166d7170d98944e88de2eccda9d30545"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.037001 24045 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:21.038883 24060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.039799 24045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:21.039891 24045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "166d7170d98944e88de2eccda9d30545"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.039963 24045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:21.052963 24045 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:21.053426 24045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:21.053560 24045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:21.053993 24045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:21.054749 24045 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:21.054791 24045 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.054836 24045 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:21.054864 24045 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.062289 24045 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.196:46669
I20240501 14:10:21.062314 24173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.196:46669 every 8 connection(s)
I20240501 14:10:21.063050 24045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 14:10:21.063556 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 24045
I20240501 14:10:21.063669 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 14:10:21.065860 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.197:0
--local_ip_for_outbound_sockets=127.6.153.197
--webserver_interface=127.6.153.197
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:21.071934 24174 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:21.072146 24174 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:21.072719 24174 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:21.109369 23590 ts_manager.cc:194] Registered new tserver with Master: 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:21.110050 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.196:47291
W20240501 14:10:21.168229 24177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:21.168399 24177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:21.168452 24177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:21.171439 24177 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:21.171495 24177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:21.171581 24177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.197
I20240501 14:10:21.174901 24177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.6.153.197
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.24177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.197
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:21.175637 24177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:21.176589 24177 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:21.183600 24183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.183874 24184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.184235 24186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:21.184679 24177 server_base.cc:1034] running on GCE node
I20240501 14:10:21.184974 24177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:21.185412 24177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:21.186564 24177 hybrid_clock.cc:648] HybridClock initialized: now 1714572621186542 us; error 27 us; skew 500 ppm
I20240501 14:10:21.188292 24177 webserver.cc:460] Webserver started at http://127.6.153.197:39413/ using document root <none> and password file <none>
I20240501 14:10:21.188776 24177 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:21.188829 24177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:21.188979 24177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:21.190414 24177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "dbeb6a07fa7a44019ae00c169c414529"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.190848 24177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "dbeb6a07fa7a44019ae00c169c414529"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.193840 24177 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:21.195729 24192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.196609 24177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:21.196702 24177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "dbeb6a07fa7a44019ae00c169c414529"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.196771 24177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:21.215706 24177 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:21.216187 24177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:21.216313 24177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:21.216751 24177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:21.217494 24177 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:21.217543 24177 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.217612 24177 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:21.217639 24177 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.225004 24177 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.197:38101
I20240501 14:10:21.225040 24305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.197:38101 every 8 connection(s)
I20240501 14:10:21.225767 24177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 14:10:21.230919 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 24177
I20240501 14:10:21.231009 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 14:10:21.233049 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.198:0
--local_ip_for_outbound_sockets=127.6.153.198
--webserver_interface=127.6.153.198
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:21.233397 24306 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:21.233595 24306 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:21.234100 24306 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:21.266891 23590 ts_manager.cc:194] Registered new tserver with Master: dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:21.267808 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.197:39693
W20240501 14:10:21.333892 24309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:21.334060 24309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:21.334098 24309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:21.337059 24309 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:21.337111 24309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:21.337196 24309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.198
I20240501 14:10:21.340553 24309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.198:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.6.153.198
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.24309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.198
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:21.341246 24309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:21.342195 24309 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:21.349617 24318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.349972 24315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.350339 24316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:21.350732 24309 server_base.cc:1034] running on GCE node
I20240501 14:10:21.351030 24309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:21.351493 24309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:21.352648 24309 hybrid_clock.cc:648] HybridClock initialized: now 1714572621352625 us; error 35 us; skew 500 ppm
I20240501 14:10:21.354224 24309 webserver.cc:460] Webserver started at http://127.6.153.198:43901/ using document root <none> and password file <none>
I20240501 14:10:21.354691 24309 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:21.354736 24309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:21.354879 24309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:21.356336 24309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.356734 24309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.359560 24309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:10:21.361354 24324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.362206 24309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:21.362296 24309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.362367 24309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:21.372781 24309 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:21.373230 24309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:21.373358 24309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:21.373797 24309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:21.374531 24309 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:21.374581 24309 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.374629 24309 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:21.374655 24309 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.381966 24309 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.198:41897
I20240501 14:10:21.382012 24437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.198:41897 every 8 connection(s)
I20240501 14:10:21.382711 24309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 14:10:21.387993 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 24309
I20240501 14:10:21.388098 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 14:10:21.390197 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.199:0
--local_ip_for_outbound_sockets=127.6.153.199
--webserver_interface=127.6.153.199
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:21.390707 24438 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:21.390887 24438 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:21.391360 24438 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:21.426343 23590 ts_manager.cc:194] Registered new tserver with Master: 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:21.427194 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.198:57847
W20240501 14:10:21.488924 24441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:21.489115 24441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:21.489156 24441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:21.492022 24441 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:21.492079 24441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:21.492144 24441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.199
I20240501 14:10:21.495577 24441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.199:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.6.153.199
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.24441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.199
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:21.496290 24441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:21.497237 24441 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:21.504633 24450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.504977 24447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.505287 24448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:21.505683 24441 server_base.cc:1034] running on GCE node
I20240501 14:10:21.505996 24441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:21.506453 24441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:21.507597 24441 hybrid_clock.cc:648] HybridClock initialized: now 1714572621507585 us; error 27 us; skew 500 ppm
I20240501 14:10:21.509198 24441 webserver.cc:460] Webserver started at http://127.6.153.199:36179/ using document root <none> and password file <none>
I20240501 14:10:21.509686 24441 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:21.509742 24441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:21.509889 24441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:21.511317 24441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "40b69f74de5a447f86c1445f8e84850d"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.511776 24441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "40b69f74de5a447f86c1445f8e84850d"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.514712 24441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20240501 14:10:21.516680 24456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.517558 24441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:21.517652 24441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "40b69f74de5a447f86c1445f8e84850d"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.517720 24441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:21.541642 24441 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:21.542112 24441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:21.542256 24441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:21.542740 24441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:21.543622 24441 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:21.543666 24441 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.543715 24441 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:21.543741 24441 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.551687 24441 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.199:35305
I20240501 14:10:21.551720 24569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.199:35305 every 8 connection(s)
I20240501 14:10:21.552536 24441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 14:10:21.554905 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 24441
I20240501 14:10:21.555011 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 14:10:21.557121 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.200:0
--local_ip_for_outbound_sockets=127.6.153.200
--webserver_interface=127.6.153.200
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:21.560686 24570 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:21.560890 24570 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:21.561334 24570 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:21.585803 23778 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
I20240501 14:10:21.618502 23590 ts_manager.cc:194] Registered new tserver with Master: 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:21.619167 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.199:38547
W20240501 14:10:21.659641 24573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:21.659823 24573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:21.659878 24573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:21.662807 24573 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:21.662866 24573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:21.662945 24573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.200
I20240501 14:10:21.666497 24573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.200:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.6.153.200
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.24573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.200
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:21.667263 24573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:21.668310 24573 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:21.675921 24580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.676259 24579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.676656 24582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:21.677064 24573 server_base.cc:1034] running on GCE node
I20240501 14:10:21.677371 24573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:21.677855 24573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:21.678999 24573 hybrid_clock.cc:648] HybridClock initialized: now 1714572621678984 us; error 31 us; skew 500 ppm
I20240501 14:10:21.680716 24573 webserver.cc:460] Webserver started at http://127.6.153.200:38521/ using document root <none> and password file <none>
I20240501 14:10:21.681201 24573 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:21.681257 24573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:21.681412 24573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:21.683048 24573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "6d832f3f723641778e60b3fd855ff98e"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.683889 24573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "6d832f3f723641778e60b3fd855ff98e"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.686985 24573 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240501 14:10:21.689002 24588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.689877 24573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:21.689972 24573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "6d832f3f723641778e60b3fd855ff98e"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.690052 24573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:21.702747 24573 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:21.703260 24573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:21.703418 24573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:21.703920 24573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:21.704720 24573 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:21.704766 24573 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.704813 24573 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:21.704840 24573 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.712198 24573 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.200:37319
I20240501 14:10:21.712240 24701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.200:37319 every 8 connection(s)
I20240501 14:10:21.712986 24573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 14:10:21.721333 24702 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:21.721542 24702 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:21.721989 24702 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:21.722404 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 24573
I20240501 14:10:21.722510 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 14:10:21.726068 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.201:0
--local_ip_for_outbound_sockets=127.6.153.201
--webserver_interface=127.6.153.201
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:21.753391 23590 ts_manager.cc:194] Registered new tserver with Master: 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319)
I20240501 14:10:21.754266 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.200:52631
I20240501 14:10:21.773381 23910 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
W20240501 14:10:21.827996 24706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:21.828158 24706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:21.828197 24706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:21.831015 24706 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:21.831063 24706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:21.831132 24706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.201
I20240501 14:10:21.834476 24706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.201:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.6.153.201
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.24706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.201
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:21.835155 24706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:21.836072 24706 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:21.843124 24712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.843454 24711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:21.843796 24714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:21.844183 24706 server_base.cc:1034] running on GCE node
I20240501 14:10:21.844508 24706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:21.844951 24706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:21.846104 24706 hybrid_clock.cc:648] HybridClock initialized: now 1714572621846092 us; error 25 us; skew 500 ppm
I20240501 14:10:21.847779 24706 webserver.cc:460] Webserver started at http://127.6.153.201:36785/ using document root <none> and password file <none>
I20240501 14:10:21.848268 24706 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:21.848318 24706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:21.848484 24706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:21.849947 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "a308c0ed71f44f57967c97239d1198f7"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.850374 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "a308c0ed71f44f57967c97239d1198f7"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.853253 24706 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240501 14:10:21.855129 24720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.856006 24706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:21.856091 24706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "a308c0ed71f44f57967c97239d1198f7"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:21.856170 24706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:21.868296 24706 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:21.868788 24706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:21.868921 24706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:21.869439 24706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:21.870291 24706 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:21.870337 24706 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.870378 24706 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:21.870406 24706 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:21.877816 24706 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.201:36409
I20240501 14:10:21.877859 24833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.201:36409 every 8 connection(s)
I20240501 14:10:21.878579 24706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 14:10:21.882187 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 24706
I20240501 14:10:21.882285 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240501 14:10:21.884349 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.202:0
--local_ip_for_outbound_sockets=127.6.153.202
--webserver_interface=127.6.153.202
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:21.886587 24834 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:21.886777 24834 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:21.887214 24834 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:21.921002 23590 ts_manager.cc:194] Registered new tserver with Master: a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409)
I20240501 14:10:21.921936 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.201:50087
I20240501 14:10:21.945539 24042 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
W20240501 14:10:21.986204 24837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:21.986369 24837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:21.986407 24837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:21.989383 24837 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:21.989435 24837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:21.989503 24837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.202
I20240501 14:10:21.992879 24837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.202:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.6.153.202
--webserver_port=0
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.24837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.202
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:21.993587 24837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:21.994539 24837 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:22.002144 24846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:22.002410 24844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:22.002774 24843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:22.003211 24837 server_base.cc:1034] running on GCE node
I20240501 14:10:22.003559 24837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:22.004040 24837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:22.005170 24837 hybrid_clock.cc:648] HybridClock initialized: now 1714572622005158 us; error 24 us; skew 500 ppm
I20240501 14:10:22.006767 24837 webserver.cc:460] Webserver started at http://127.6.153.202:39461/ using document root <none> and password file <none>
I20240501 14:10:22.007246 24837 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:22.007300 24837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:22.007479 24837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 14:10:22.008947 24837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/instance:
uuid: "c958143102d04279ac25aa56214d0d4b"
format_stamp: "Formatted at 2024-05-01 14:10:22 on dist-test-slave-cmf0"
I20240501 14:10:22.009359 24837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal/instance:
uuid: "c958143102d04279ac25aa56214d0d4b"
format_stamp: "Formatted at 2024-05-01 14:10:22 on dist-test-slave-cmf0"
I20240501 14:10:22.012244 24837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 14:10:22.014160 24852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:22.015102 24837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:22.015190 24837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "c958143102d04279ac25aa56214d0d4b"
format_stamp: "Formatted at 2024-05-01 14:10:22 on dist-test-slave-cmf0"
I20240501 14:10:22.015264 24837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:22.033270 24837 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:22.033746 24837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:22.033870 24837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:22.034300 24837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:22.035051 24837 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:22.035095 24837 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:22.035138 24837 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:22.035166 24837 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:22.042472 24837 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.202:34929
I20240501 14:10:22.042512 24965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.202:34929 every 8 connection(s)
I20240501 14:10:22.043241 24837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240501 14:10:22.049647 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 24837
I20240501 14:10:22.049775 6759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal/instance
I20240501 14:10:22.050895 24966 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:22.051096 24966 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:22.051615 24966 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:22.083479 23590 ts_manager.cc:194] Registered new tserver with Master: c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929)
I20240501 14:10:22.084097 23590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.202:50185
I20240501 14:10:22.086542 6759 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240501 14:10:22.094879 6759 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240501 14:10:22.094945 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 24706
I20240501 14:10:22.100078 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 24837
I20240501 14:10:22.104225 6759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxMccU0/build/debug/bin/kudu with pid 24573
I20240501 14:10:22.112001 24174 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
I20240501 14:10:22.269943 24306 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
I20240501 14:10:22.429517 24438 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
I20240501 14:10:22.621129 24570 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
I20240501 14:10:25.861666 23590 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37374:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:25.864722 23590 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 14:10:25.882567 23844 tablet_service.cc:1467] Processing CreateTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.882570 23974 tablet_service.cc:1467] Processing CreateTablet for tablet 76fbefe89b8344e1a09101d404f32d9e (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.882571 23975 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.882571 23976 tablet_service.cc:1467] Processing CreateTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.883545 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.883595 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.884039 23842 tablet_service.cc:1467] Processing CreateTablet for tablet 76fbefe89b8344e1a09101d404f32d9e (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.884711 23976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.885208 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.885978 24372 tablet_service.cc:1467] Processing CreateTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.886301 23842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.886756 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.887614 24371 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.887830 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.889083 23712 tablet_service.cc:1467] Processing CreateTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.889823 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.890872 23711 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.891095 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.891568 24370 tablet_service.cc:1467] Processing CreateTablet for tablet 76fbefe89b8344e1a09101d404f32d9e (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.891433 24238 tablet_service.cc:1467] Processing CreateTablet for tablet 76fbefe89b8344e1a09101d404f32d9e (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.891779 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.892102 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.895435 23975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.895450 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.895571 23710 tablet_service.cc:1467] Processing CreateTablet for tablet 76fbefe89b8344e1a09101d404f32d9e (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.895781 23710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.895905 24993 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:25.897879 24993 tablet_bootstrap.cc:654] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.898102 24240 tablet_service.cc:1467] Processing CreateTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.898324 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.898811 24993 log.cc:824] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:25.900411 24993 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:25.900418 24239 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.900563 24993 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240501 14:10:25.900624 24239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.903115 24998 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:25.903699 24993 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.903959 24993 raft_consensus.cc:374] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.904035 24993 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.904479 24993 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.905387 24993 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 14:10:25.905411 24998 tablet_bootstrap.cc:654] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.906123 24998 log.cc:824] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:25.905965 24108 tablet_service.cc:1467] Processing CreateTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.906731 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.906818 24993 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:25.907639 24993 tablet_bootstrap.cc:654] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.907588 24107 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.907743 24998 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:25.907812 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.907891 24998 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240501 14:10:25.908613 24993 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:25.908682 24993 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:25.909137 24993 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.909237 24993 raft_consensus.cc:374] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.909262 24993 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.909399 24993 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.909718 24993 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:25.909876 24993 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:25.910687 24993 tablet_bootstrap.cc:654] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.911080 24998 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.911280 24998 raft_consensus.cc:374] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.911347 24998 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.911607 24106 tablet_service.cc:1467] Processing CreateTablet for tablet 76fbefe89b8344e1a09101d404f32d9e (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.911834 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.911844 24993 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:25.911814 24998 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.911907 24993 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:25.912288 24993 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.912384 24993 raft_consensus.cc:374] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.912410 24993 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.912518 24993 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.912632 24998 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 14:10:25.912768 24993 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:25.912938 24998 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:25.913611 24998 tablet_bootstrap.cc:654] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.914592 24998 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:25.914696 24998 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:25.915302 25006 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:25.915231 24998 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.915395 24998 raft_consensus.cc:374] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.915454 24998 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.915570 24998 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.915947 24998 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:25.916167 24998 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:25.916981 24998 tablet_bootstrap.cc:654] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.917289 25006 tablet_bootstrap.cc:654] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.918318 25006 log.cc:824] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:25.919472 24998 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:25.919600 24998 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:10:25.920087 24995 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:25.920044 24998 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.920148 24998 raft_consensus.cc:374] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.920197 24998 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.920243 25006 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:25.920361 24998 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.920544 25002 raft_consensus.cc:482] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:25.920675 25002 raft_consensus.cc:504] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.922075 25002 leader_election.cc:288] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:25.922123 25006 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20240501 14:10:25.923882 24504 tablet_service.cc:1467] Processing CreateTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.924755 24504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.921592 24996 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:25.925082 25006 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.925259 25006 raft_consensus.cc:374] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.925299 25006 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.925693 25006 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.926419 25006 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20240501 14:10:25.926555 24996 tablet_bootstrap.cc:654] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.926729 25006 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:25.927258 24996 log.cc:824] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:25.927552 25006 tablet_bootstrap.cc:654] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.927611 24503 tablet_service.cc:1467] Processing CreateTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.927839 24503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.928507 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:25.929562 24994 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
W20240501 14:10:25.930243 24326 leader_election.cc:341] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:10:25.930502 24995 tablet_bootstrap.cc:654] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.931291 24994 tablet_bootstrap.cc:654] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.931586 24502 tablet_service.cc:1467] Processing CreateTablet for tablet 76fbefe89b8344e1a09101d404f32d9e (DEFAULT_TABLE table=rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:25.931802 24502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:25.932006 24994 log.cc:824] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:25.933678 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:25.933893 24260 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.934800 24995 log.cc:824] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:25.934636 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
W20240501 14:10:25.936205 24328 leader_election.cc:341] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:10:25.936105 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:25.937341 24998 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.018s user 0.001s sys 0.000s
I20240501 14:10:25.937379 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:25.937562 24128 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.937688 25018 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:25.937709 24994 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:25.937852 24994 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
W20240501 14:10:25.937956 24327 leader_election.cc:341] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:10:25.939553 24995 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:25.939615 25018 tablet_bootstrap.cc:654] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.939697 24995 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.002s
I20240501 14:10:25.940253 25018 log.cc:824] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:25.941139 24994 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.941334 24994 raft_consensus.cc:374] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.941385 24994 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.941701 24996 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:25.941854 24996 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20240501 14:10:25.941911 24995 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.942016 24994 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.942164 24995 raft_consensus.cc:374] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.942307 24995 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.942415 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:25.942646 23864 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.942951 24328 leader_election.cc:302] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters: 24d90b55560e45feb5fe4b9482317a4c, 40b69f74de5a447f86c1445f8e84850d, 8ebe0a098b0f4ed4861a5605717fecbd
I20240501 14:10:25.943176 25002 raft_consensus.cc:2781] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:25.943243 25002 raft_consensus.cc:482] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:25.943279 25002 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.943547 24995 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.944015 25018 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:25.944144 25018 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20240501 14:10:25.944080 24996 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.944240 24996 raft_consensus.cc:374] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.944281 24996 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.944231 25002 raft_consensus.cc:504] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.944391 24995 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20240501 14:10:25.944753 24995 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:25.944746 25002 leader_election.cc:288] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Requested vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:25.945251 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:25.945343 24128 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.945524 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:25.945580 24996 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.945761 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:25.945855 24260 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.946246 24128 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.946333 24996 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240501 14:10:25.946349 25018 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.946501 25018 raft_consensus.cc:374] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.946544 25018 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.946655 24996 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:25.946934 25018 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.947194 24260 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.947407 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:25.947515 24996 tablet_bootstrap.cc:654] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.947690 25018 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20240501 14:10:25.947925 24995 tablet_bootstrap.cc:654] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.948637 25006 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:25.948693 25018 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:25.948710 25006 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
W20240501 14:10:25.949043 24327 leader_election.cc:341] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:10:25.949200 25006 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.949299 25006 raft_consensus.cc:374] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.949331 25006 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.949474 25018 tablet_bootstrap.cc:654] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.949446 25006 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.949702 25006 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:25.949836 25006 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
W20240501 14:10:25.949877 24326 leader_election.cc:341] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:10:25.950459 24996 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:25.950640 25006 tablet_bootstrap.cc:654] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.950534 24996 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:10:25.950865 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:25.950966 23864 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.951170 24996 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.951268 24996 raft_consensus.cc:374] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.951297 24996 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.951447 24996 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.951469 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:25.951591 23996 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.951797 24996 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:25.951925 23864 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.951953 24996 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:25.952508 23996 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.952579 24328 leader_election.cc:302] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters: 24d90b55560e45feb5fe4b9482317a4c, 40b69f74de5a447f86c1445f8e84850d
I20240501 14:10:25.952780 24995 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:25.952863 24995 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240501 14:10:25.952942 24996 tablet_bootstrap.cc:654] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.953066 25002 raft_consensus.cc:2781] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:25.953148 25002 raft_consensus.cc:686] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Becoming Leader. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Running, Role: LEADER
I20240501 14:10:25.953392 25002 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.953497 24994 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.016s user 0.004s sys 0.000s
I20240501 14:10:25.953487 24995 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.953593 24995 raft_consensus.cc:374] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.953624 24995 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.953787 24994 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:25.954275 25018 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:25.954340 25018 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:10:25.954610 24994 tablet_bootstrap.cc:654] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.954802 25018 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.954905 25018 raft_consensus.cc:374] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.954933 25018 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.955042 25018 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.955375 25018 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:25.955564 25018 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:25.955935 24994 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:25.956068 24994 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 14:10:25.956379 25018 tablet_bootstrap.cc:654] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.956566 24994 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.956732 24994 raft_consensus.cc:374] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.956812 24994 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.956979 24994 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.956904 23590 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198). New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:25.957365 24994 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:25.957502 24994 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:25.957669 24995 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.958027 24995 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20240501 14:10:25.958277 24994 tablet_bootstrap.cc:654] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.958869 24995 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:25.959657 24996 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:25.959741 24996 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240501 14:10:25.959844 25006 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:25.959928 25006 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 14:10:25.960134 24996 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.960331 25006 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.960444 25006 raft_consensus.cc:374] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.960479 25006 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.960601 25006 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:25.960842 25006 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:25.961024 24995 tablet_bootstrap.cc:654] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:25.961108 24996 raft_consensus.cc:374] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.961189 24996 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.961365 24996 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.961710 24996 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:25.962028 25018 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:25.962109 25018 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240501 14:10:25.962613 25018 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.962718 25018 raft_consensus.cc:374] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.962752 25018 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.962888 25018 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.963128 25018 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:25.963142 24995 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:25.963212 24995 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 14:10:25.963733 24995 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.963804 24994 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:25.963850 24995 raft_consensus.cc:374] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.963876 24994 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240501 14:10:25.963884 24995 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.963986 24995 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.964242 24995 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:25.964354 24994 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.964454 24994 raft_consensus.cc:374] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:25.964495 24994 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:25.964607 24994 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.964875 24994 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240501 14:10:25.979311 24307 tablet.cc:2369] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:10:25.980772 23911 tablet.cc:2369] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:10:25.989544 25002 raft_consensus.cc:482] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:25.989641 25002 raft_consensus.cc:504] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.990164 25002 leader_election.cc:288] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:25.990301 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:25.990314 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:25.990417 23996 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.990515 23732 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.990515 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:25.990516 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:25.990620 23864 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.990620 24260 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.990988 24328 leader_election.cc:302] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:25.991008 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:25.991144 25002 raft_consensus.cc:2781] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:25.991183 25002 raft_consensus.cc:482] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:25.991220 25002 raft_consensus.cc:3037] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.991204 24524 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.991204 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:25.991305 24128 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:25.992121 25002 raft_consensus.cc:504] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.992580 25002 leader_election.cc:288] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Requested vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:25.992727 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:25.992811 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:25.992820 23732 raft_consensus.cc:3037] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.992888 23864 raft_consensus.cc:3037] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.992976 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:25.992988 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:25.993062 23996 raft_consensus.cc:3037] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.993074 24260 raft_consensus.cc:3037] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.993750 23864 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.993763 23732 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.993826 23996 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.993893 24260 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.994232 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:25.994339 24128 raft_consensus.cc:3037] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.994328 24328 leader_election.cc:302] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:25.994351 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777ba3b6be404d12bb0b219c66da71a6" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:25.994438 24524 raft_consensus.cc:3037] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:25.994501 25002 raft_consensus.cc:2781] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:25.994580 25002 raft_consensus.cc:686] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Becoming Leader. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Running, Role: LEADER
I20240501 14:10:25.994709 25002 consensus_queue.cc:237] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:25.995132 24128 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.995402 24524 raft_consensus.cc:2454] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:25.996598 23588 catalog_manager.cc:5484] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198). New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:26.022246 25030 raft_consensus.cc:482] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:26.022358 25030 raft_consensus.cc:504] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.023643 25030 leader_election.cc:288] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.028261 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:26.028386 24128 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.028761 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:26.028882 23864 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.028955 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:26.029140 24392 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.029628 23929 leader_election.cc:302] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:26.029870 25030 raft_consensus.cc:2781] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:26.029920 25030 raft_consensus.cc:482] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:26.029980 25030 raft_consensus.cc:3037] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.030212 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:26.030344 24524 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.030860 25030 raft_consensus.cc:504] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.031466 25030 leader_election.cc:288] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 election: Requested vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.031697 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:26.031790 23864 raft_consensus.cc:3037] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.031822 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:26.031901 24392 raft_consensus.cc:3037] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.031872 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:26.031992 23732 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.032213 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:26.032313 23731 raft_consensus.cc:3037] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.032639 23864 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.032714 24392 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.032691 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:26.032804 24260 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.032946 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:26.033043 24128 raft_consensus.cc:3037] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.033202 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:26.033255 23731 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.033286 24524 raft_consensus.cc:3037] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.033464 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fc754b4d0f144c4ba0bd9a1b77f7bb4" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:26.033558 24259 raft_consensus.cc:3037] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.033865 24128 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.033879 23932 leader_election.cc:302] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:26.034092 24524 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.034093 25030 raft_consensus.cc:2781] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:26.034178 25030 raft_consensus.cc:686] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Becoming Leader. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Running, Role: LEADER
I20240501 14:10:26.034489 24259 raft_consensus.cc:2454] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.034488 25030 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
W20240501 14:10:26.034857 23779 tablet.cc:2369] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 14:10:26.037652 23588 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195). New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
W20240501 14:10:26.055821 24571 tablet.cc:2369] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:10:26.066630 24175 tablet.cc:2369] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:10:26.136438 24439 tablet.cc:2369] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:10:26.144492 25048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:26.144678 25048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:26.144873 24043 tablet.cc:2369] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 14:10:26.147603 25048 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 14:10:26.149808 25034 raft_consensus.cc:482] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:26.149921 25034 raft_consensus.cc:504] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.151175 25034 leader_election.cc:288] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:26.156224 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:26.156359 23731 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 0.
I20240501 14:10:26.157588 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:26.159368 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:26.159543 24128 raft_consensus.cc:2379] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 40b69f74de5a447f86c1445f8e84850d in current term 1: Already voted for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in this term.
I20240501 14:10:26.159982 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:26.160158 24259 raft_consensus.cc:2379] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 40b69f74de5a447f86c1445f8e84850d in current term 1: Already voted for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in this term.
I20240501 14:10:26.161610 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:26.161789 23996 raft_consensus.cc:2379] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 40b69f74de5a447f86c1445f8e84850d in current term 1: Already voted for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in this term.
I20240501 14:10:26.162101 24460 leader_election.cc:302] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 40b69f74de5a447f86c1445f8e84850d; no voters: 166d7170d98944e88de2eccda9d30545, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529
I20240501 14:10:26.162302 25034 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.162647 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:26.162834 23864 raft_consensus.cc:2379] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 40b69f74de5a447f86c1445f8e84850d in current term 1: Already voted for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in this term.
I20240501 14:10:26.163194 25034 raft_consensus.cc:2726] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Using table 'rebalance_test_table_0'
I20240501 14:10:26.228674 24524 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.228678 24128 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.229624 23731 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.229722 23996 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.230394 24259 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.230382 24999 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.230772 24999 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.231158 23864 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.232190 25002 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.232403 23864 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.232465 25002 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.232698 25002 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.232800 23732 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.232851 23996 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.232933 25002 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.233198 24259 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.233839 23732 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.233981 24127 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.235224 24524 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.237290 24999 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.237556 24999 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.237819 24999 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.237942 25087 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.238209 24999 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.240031 25088 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.262172 25085 mvcc.cc:205] Tried to move back new op lower bound from 7022889477022609408 to 7022889476077359104. Current Snapshot: MvccSnapshot[applied={T|T < 7022889477022609408}]
I20240501 14:10:26.264487 25101 mvcc.cc:205] Tried to move back new op lower bound from 7022889477067960320 to 7022889475908927488. Current Snapshot: MvccSnapshot[applied={T|T < 7022889477039882240}]
I20240501 14:10:26.266937 25099 mvcc.cc:205] Tried to move back new op lower bound from 7022889477045575680 to 7022889476077359104. Current Snapshot: MvccSnapshot[applied={T|T < 7022889477022609408}]
I20240501 14:10:26.276197 25104 mvcc.cc:205] Tried to move back new op lower bound from 7022889477045575680 to 7022889476077359104. Current Snapshot: MvccSnapshot[applied={T|T < 7022889477022609408}]
I20240501 14:10:26.278156 24523 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.278626 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.278751 24127 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.294679 25113 mvcc.cc:205] Tried to move back new op lower bound from 7022889477067960320 to 7022889475908927488. Current Snapshot: MvccSnapshot[applied={T|T < 7022889477039882240}]
I20240501 14:10:26.314936 25109 mvcc.cc:205] Tried to move back new op lower bound from 7022889477067960320 to 7022889475908927488. Current Snapshot: MvccSnapshot[applied={T|T < 7022889477039882240}]
I20240501 14:10:26.279172 25024 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.324988 25030 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.326957 23731 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:10:26.327414 24260 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:10:26.327471 23864 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 14:10:26.328742 25127 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.329172 25130 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.332863 25024 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.333163 25024 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.345518 25098 mvcc.cc:205] Tried to move back new op lower bound from 7022889477039882240 to 7022889475908927488. Current Snapshot: MvccSnapshot[applied={T|T < 7022889477039882240}]
INSERT report
rows total: 2000
time total: 175.858 ms
time per row: 0.0879291 ms
I20240501 14:10:26.374898 23589 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37374:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:26.375401 23589 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 14:10:26.380923 24107 tablet_service.cc:1467] Processing CreateTablet for tablet ba3621a3f5df43eca96826da75fc31d0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.381235 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.382201 24372 tablet_service.cc:1467] Processing CreateTablet for tablet ba3621a3f5df43eca96826da75fc31d0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.382464 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.383360 25006 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:26.384151 25006 tablet_bootstrap.cc:654] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.384461 24993 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:26.385171 24993 tablet_bootstrap.cc:654] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.385394 25006 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:26.385463 25006 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:26.385886 25006 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.386029 25006 raft_consensus.cc:374] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.386065 25006 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.386103 24239 tablet_service.cc:1467] Processing CreateTablet for tablet ba3621a3f5df43eca96826da75fc31d0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.386191 25006 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.386332 24239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.386461 25006 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.388273 24998 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:26.388672 23975 tablet_service.cc:1467] Processing CreateTablet for tablet 34a278486c8b447d8dc4c3241743de60 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.388903 23975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.389011 23844 tablet_service.cc:1467] Processing CreateTablet for tablet ba3621a3f5df43eca96826da75fc31d0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.389067 24998 tablet_bootstrap.cc:654] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.389235 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.389698 23974 tablet_service.cc:1467] Processing CreateTablet for tablet ba3621a3f5df43eca96826da75fc31d0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.389896 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.390857 24996 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:26.391302 23843 tablet_service.cc:1467] Processing CreateTablet for tablet d25ef07189ac43749efc549299dc89f8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.391459 24994 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:26.391667 23976 tablet_service.cc:1467] Processing CreateTablet for tablet d25ef07189ac43749efc549299dc89f8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.391746 23842 tablet_service.cc:1467] Processing CreateTablet for tablet 34a278486c8b447d8dc4c3241743de60 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.391846 23976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.392210 24996 tablet_bootstrap.cc:654] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.393127 24238 tablet_service.cc:1467] Processing CreateTablet for tablet 34a278486c8b447d8dc4c3241743de60 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.393182 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.393326 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.393554 23711 tablet_service.cc:1467] Processing CreateTablet for tablet ba3621a3f5df43eca96826da75fc31d0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.393792 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.395069 23842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.395699 24994 tablet_bootstrap.cc:654] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.395762 23712 tablet_service.cc:1467] Processing CreateTablet for tablet d25ef07189ac43749efc549299dc89f8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.395959 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.396616 24994 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:26.396682 24994 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:10:26.397068 24994 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.397176 24994 raft_consensus.cc:374] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.397208 24994 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.397315 24994 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.397585 24994 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.397747 24994 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:26.397711 23710 tablet_service.cc:1467] Processing CreateTablet for tablet 34a278486c8b447d8dc4c3241743de60 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.397890 23710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.398011 24371 tablet_service.cc:1467] Processing CreateTablet for tablet 34a278486c8b447d8dc4c3241743de60 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.398221 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.398501 24994 tablet_bootstrap.cc:654] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.399474 24995 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:26.399848 24994 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:26.399932 24994 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:26.400236 24995 tablet_bootstrap.cc:654] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.400332 24994 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.400450 24994 raft_consensus.cc:374] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.400485 24994 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.400592 24994 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.400854 24994 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.401008 24994 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:26.401381 24504 tablet_service.cc:1467] Processing CreateTablet for tablet ba3621a3f5df43eca96826da75fc31d0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.401652 24504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.401729 24994 tablet_bootstrap.cc:654] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.402702 24107 tablet_service.cc:1467] Processing CreateTablet for tablet d25ef07189ac43749efc549299dc89f8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.402928 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.403793 24503 tablet_service.cc:1467] Processing CreateTablet for tablet 34a278486c8b447d8dc4c3241743de60 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.404003 24503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.404873 24239 tablet_service.cc:1467] Processing CreateTablet for tablet d25ef07189ac43749efc549299dc89f8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.405069 24239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.405148 24108 tablet_service.cc:1467] Processing CreateTablet for tablet 34a278486c8b447d8dc4c3241743de60 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.405217 25006 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:26.405341 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.405993 25006 tablet_bootstrap.cc:654] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.407231 24994 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:26.407301 24994 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:10:26.407280 24502 tablet_service.cc:1467] Processing CreateTablet for tablet d25ef07189ac43749efc549299dc89f8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.407511 24502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.407737 24994 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.407866 24994 raft_consensus.cc:374] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.407893 24994 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.408005 24994 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.408244 24994 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.409286 25006 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:26.409360 25006 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:10:26.409777 25006 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.409876 25006 raft_consensus.cc:374] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.409904 25006 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.410034 25006 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.410275 25006 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.410432 25006 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:26.411195 24372 tablet_service.cc:1467] Processing CreateTablet for tablet d25ef07189ac43749efc549299dc89f8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.411437 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.411582 25018 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:26.412343 25018 tablet_bootstrap.cc:654] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.413089 25006 tablet_bootstrap.cc:654] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.413409 24995 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:26.413484 24995 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20240501 14:10:26.413740 25018 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:26.413821 25018 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:26.413941 24995 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.414048 24995 raft_consensus.cc:374] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.414104 24995 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.414217 24995 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.414223 25018 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.414317 25018 raft_consensus.cc:374] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.414345 25018 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.414445 25018 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.414517 24995 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:26.414657 24995 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:26.414700 25018 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.414845 25018 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:26.415470 24995 tablet_bootstrap.cc:654] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.416716 25018 tablet_bootstrap.cc:654] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.416900 25023 raft_consensus.cc:482] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:26.416996 25023 raft_consensus.cc:504] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.417708 24995 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:26.417768 24995 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 14:10:26.418169 25018 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:26.418234 25018 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:10:26.418257 24995 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.418368 24995 raft_consensus.cc:374] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.418399 24995 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.418485 24995 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.418535 25023 leader_election.cc:288] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:26.418606 25018 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.418704 25018 raft_consensus.cc:374] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.418721 24995 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:26.418730 25018 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.418859 24995 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:26.418833 25018 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.419095 25018 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.419220 25018 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:26.419677 24995 tablet_bootstrap.cc:654] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.421391 25018 tablet_bootstrap.cc:654] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.422444 24995 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:26.422514 24995 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:10:26.422901 24995 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.423008 24995 raft_consensus.cc:374] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.423040 24995 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.423146 24995 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.423424 24995 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.424654 25018 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:26.424722 25018 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:10:26.425088 25018 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.425184 25018 raft_consensus.cc:374] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.425210 25018 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.425307 25018 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.425551 25018 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.428112 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:26.428599 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:26.428709 23864 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
W20240501 14:10:26.428790 23665 leader_election.cc:341] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:10:26.430874 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
W20240501 14:10:26.431243 23666 leader_election.cc:341] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:10:26.432399 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
W20240501 14:10:26.432770 23668 leader_election.cc:341] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): Illegal state: must be running to vote when last-logged opid is not known
I20240501 14:10:26.433018 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:26.433125 24523 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.433293 24998 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:26.433382 24998 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.000s
I20240501 14:10:26.433583 24993 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:26.433588 24996 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:26.433658 24993 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.000s
I20240501 14:10:26.433658 24996 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.001s
I20240501 14:10:26.433769 24998 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.433878 24998 raft_consensus.cc:374] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.433912 24998 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.434026 24998 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.434022 24993 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.434052 24996 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.434136 24993 raft_consensus.cc:374] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.434149 24996 raft_consensus.cc:374] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.434166 24993 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.434182 24996 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.434284 24998 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.434280 24993 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.434281 24996 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.434392 25006 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:26.434441 24998 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:26.434465 25006 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20240501 14:10:26.434535 24996 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.434546 24993 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.434693 24996 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:26.434691 24993 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:26.434870 25006 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.434989 25006 raft_consensus.cc:374] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.435019 25006 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.435204 24998 tablet_bootstrap.cc:654] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.435142 25006 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.435454 25006 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.435534 24993 tablet_bootstrap.cc:654] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.436609 24996 tablet_bootstrap.cc:654] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.438182 24996 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:26.438302 24996 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:10:26.438709 24996 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.438866 24996 raft_consensus.cc:374] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.438931 24996 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.439106 24996 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.439399 24996 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.439989 24996 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:26.440832 24996 tablet_bootstrap.cc:654] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.440972 24993 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:26.441063 24993 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:10:26.441148 24998 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:26.441219 24998 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240501 14:10:26.441432 24993 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.441532 24993 raft_consensus.cc:374] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.441560 24993 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.441622 24998 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.441753 24998 raft_consensus.cc:374] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.441728 24993 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.441805 24998 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.442157 24998 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.442231 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:26.442345 24127 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.442612 23667 leader_election.cc:302] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 40b69f74de5a447f86c1445f8e84850d; no voters: 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529
I20240501 14:10:26.442837 24998 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:26.443001 24998 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:26.443095 25037 raft_consensus.cc:2781] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:26.443152 25037 raft_consensus.cc:482] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:26.443194 25037 raft_consensus.cc:3037] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.443393 24996 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:26.443464 24996 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:10:26.443876 24998 tablet_bootstrap.cc:654] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.443823 24996 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.443919 24996 raft_consensus.cc:374] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.443948 24996 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.444054 24996 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.444307 24996 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.444514 25037 raft_consensus.cc:504] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.444700 24993 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:10:26.444895 24993 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:26.445125 25037 leader_election.cc:288] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 election: Requested vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:26.445458 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:26.445562 24259 raft_consensus.cc:3037] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.445684 24993 tablet_bootstrap.cc:654] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.445690 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:26.445703 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:26.445791 23864 raft_consensus.cc:3037] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.445791 23995 raft_consensus.cc:3037] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.446409 24259 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.446624 23864 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.446700 23995 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.446746 24998 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:26.446807 24998 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 14:10:26.446933 23668 leader_election.cc:302] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:26.447194 25037 raft_consensus.cc:2781] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:26.447180 24998 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.447279 24998 raft_consensus.cc:374] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.447280 25037 raft_consensus.cc:686] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Becoming Leader. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Running, Role: LEADER
I20240501 14:10:26.447311 24998 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.447320 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:26.447427 24392 raft_consensus.cc:3037] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.447465 24998 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.447584 25037 consensus_queue.cc:237] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.447721 24998 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.448282 24392 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.448274 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:26.448380 24523 raft_consensus.cc:3037] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.448691 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3621a3f5df43eca96826da75fc31d0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:26.448781 24127 raft_consensus.cc:3037] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.449038 24993 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:26.449117 24993 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:10:26.449306 24523 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.449487 24993 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.449584 24993 raft_consensus.cc:374] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.449616 24993 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.449676 24127 raft_consensus.cc:2454] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.449724 24993 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.449975 24993 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.451717 23587 catalog_manager.cc:5484] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193). New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:26.461172 25004 raft_consensus.cc:482] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:26.461258 25004 raft_consensus.cc:504] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.462400 25004 leader_election.cc:288] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.467582 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:26.467695 23864 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:26.468008 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:26.468113 24127 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:26.468837 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:26.468945 24523 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:26.469274 24195 leader_election.cc:302] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 40b69f74de5a447f86c1445f8e84850d, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:26.469328 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:26.469434 23731 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:26.469491 25004 raft_consensus.cc:2781] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:26.469538 25004 raft_consensus.cc:482] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:26.469571 25004 raft_consensus.cc:3037] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.470252 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:26.470357 24392 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:26.470402 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:26.470381 25004 raft_consensus.cc:504] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.470494 23995 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:26.470886 25004 leader_election.cc:288] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 election: Requested vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.471076 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:26.471138 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:26.471170 23731 raft_consensus.cc:3037] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.471140 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:26.471223 24523 raft_consensus.cc:3037] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.471235 23864 raft_consensus.cc:3037] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.471266 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:26.471534 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:26.471611 24392 raft_consensus.cc:3037] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.472005 24523 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:26.472005 23731 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:26.472069 23864 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:26.472348 24392 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:26.472537 24127 raft_consensus.cc:3037] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.472541 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34a278486c8b447d8dc4c3241743de60" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:26.472585 24195 leader_election.cc:302] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 40b69f74de5a447f86c1445f8e84850d, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:26.472626 23995 raft_consensus.cc:3037] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.472738 25004 raft_consensus.cc:2781] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:26.472860 25004 raft_consensus.cc:686] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Becoming Leader. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Running, Role: LEADER
I20240501 14:10:26.473065 25004 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.473320 24127 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:26.473321 23995 raft_consensus.cc:2454] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:26.475116 23587 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197). New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:26.484583 25037 raft_consensus.cc:482] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:26.484665 25037 raft_consensus.cc:504] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.485149 25037 leader_election.cc:288] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.485309 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:26.485414 24127 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.485414 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:26.485410 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:26.485514 23995 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.485518 23864 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.485580 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:26.485599 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:26.485668 24259 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.485692 24392 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.485893 23666 leader_election.cc:302] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:26.486037 25037 raft_consensus.cc:2781] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:26.486083 25037 raft_consensus.cc:482] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:26.486124 25037 raft_consensus.cc:3037] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.486115 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:26.486231 24523 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.486896 25037 raft_consensus.cc:504] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.487425 25037 leader_election.cc:288] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 election: Requested vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.487731 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:26.487763 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:26.487828 24127 raft_consensus.cc:3037] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.487854 24392 raft_consensus.cc:3037] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.487897 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:26.487987 23864 raft_consensus.cc:3037] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.487987 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:26.488066 24259 raft_consensus.cc:3037] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.488651 24127 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.488694 24392 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.488783 23864 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.488860 24259 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.488962 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:26.489048 23995 raft_consensus.cc:3037] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.489151 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25ef07189ac43749efc549299dc89f8" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:26.489248 24523 raft_consensus.cc:3037] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.489270 23666 leader_election.cc:302] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:26.489467 25037 raft_consensus.cc:2781] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:26.489543 25037 raft_consensus.cc:686] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Becoming Leader. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Running, Role: LEADER
I20240501 14:10:26.489671 25037 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.489833 23995 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.490092 24523 raft_consensus.cc:2454] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.491541 23587 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193). New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
W20240501 14:10:26.593812 25191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:26.594012 25191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:26.596990 25191 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 14:10:26.666939 24259 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.667171 24523 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.667709 25037 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.667797 24392 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.668473 25037 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.668695 24259 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.669276 24127 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.669652 23864 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.669909 25023 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.670490 23995 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.670521 24524 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.671998 24391 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.672086 23995 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.672463 25219 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.672528 25004 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.673087 23864 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.673451 24392 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.673846 25004 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.674858 24127 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.675603 25188 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.675899 25188 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.676816 25219 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.683939 24524 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.684232 25023 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.689962 23731 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.690331 25023 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.690614 25023 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.690914 25023 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.696461 25004 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.696838 25004 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.739151 23995 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.739632 25037 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.739776 23864 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.740175 25037 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:26.742015 24127 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:26.743806 25219 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 157.348 ms
time per row: 0.0786739 ms
I20240501 14:10:26.804554 23587 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37374:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:26.805069 23587 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 14:10:26.811256 24108 tablet_service.cc:1467] Processing CreateTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.811604 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.813191 24107 tablet_service.cc:1467] Processing CreateTablet for tablet bb091eaf92de4f9bbf110f489a9f21bd (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.813385 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb091eaf92de4f9bbf110f489a9f21bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.813557 25006 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:26.814066 24372 tablet_service.cc:1467] Processing CreateTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.814327 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.814363 25006 tablet_bootstrap.cc:654] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.815258 24106 tablet_service.cc:1467] Processing CreateTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.815492 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.815574 25006 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:26.815654 25006 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:26.816337 25006 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.816453 25006 raft_consensus.cc:374] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.816483 25006 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.816504 24371 tablet_service.cc:1467] Processing CreateTablet for tablet bb091eaf92de4f9bbf110f489a9f21bd (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.816602 25006 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.816699 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb091eaf92de4f9bbf110f489a9f21bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.816877 25006 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.817323 24370 tablet_service.cc:1467] Processing CreateTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.817533 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.818205 24993 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:26.818843 24993 tablet_bootstrap.cc:654] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.820001 24993 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:26.820081 24993 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:26.820544 24993 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.820653 24993 raft_consensus.cc:374] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.820686 24993 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.820796 24993 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.821132 24993 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.821151 23710 tablet_service.cc:1467] Processing CreateTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.821305 24993 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:26.821432 23710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.821976 24239 tablet_service.cc:1467] Processing CreateTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.822036 24238 tablet_service.cc:1467] Processing CreateTablet for tablet bb091eaf92de4f9bbf110f489a9f21bd (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.822104 24993 tablet_bootstrap.cc:654] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.822209 24239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.822445 24240 tablet_service.cc:1467] Processing CreateTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.822649 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.822798 24503 tablet_service.cc:1467] Processing CreateTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.823057 24503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.823578 23711 tablet_service.cc:1467] Processing CreateTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.823801 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.824249 24995 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:26.824438 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb091eaf92de4f9bbf110f489a9f21bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.825001 24995 tablet_bootstrap.cc:654] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.825189 24502 tablet_service.cc:1467] Processing CreateTablet for tablet bb091eaf92de4f9bbf110f489a9f21bd (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.825400 24502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb091eaf92de4f9bbf110f489a9f21bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.825621 24998 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:26.825762 23712 tablet_service.cc:1467] Processing CreateTablet for tablet bb091eaf92de4f9bbf110f489a9f21bd (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.825955 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb091eaf92de4f9bbf110f489a9f21bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.826423 24998 tablet_bootstrap.cc:654] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.826594 25006 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:26.827234 24504 tablet_service.cc:1467] Processing CreateTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.827428 25006 tablet_bootstrap.cc:654] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.827472 24504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.828110 24998 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:26.828178 24998 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 14:10:26.828543 24998 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.828641 24998 raft_consensus.cc:374] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.828668 24998 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.828773 24998 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.828867 23976 tablet_service.cc:1467] Processing CreateTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.828948 23975 tablet_service.cc:1467] Processing CreateTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.829046 24998 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.829098 23976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.829188 24998 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:26.829275 23975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.829488 23974 tablet_service.cc:1467] Processing CreateTablet for tablet bb091eaf92de4f9bbf110f489a9f21bd (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.829696 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb091eaf92de4f9bbf110f489a9f21bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.829970 24998 tablet_bootstrap.cc:654] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.830309 25018 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:26.831056 25018 tablet_bootstrap.cc:654] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.831518 24996 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:26.832051 23842 tablet_service.cc:1467] Processing CreateTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.832294 24996 tablet_bootstrap.cc:654] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.832302 23842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.832989 23844 tablet_service.cc:1467] Processing CreateTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.833205 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.833577 24995 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:26.833649 24995 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:10:26.834038 24995 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.834142 24995 raft_consensus.cc:374] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.834173 24995 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.834259 24994 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:26.834283 24995 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.834546 24995 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.834692 24995 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:26.835069 24994 tablet_bootstrap.cc:654] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.835084 23843 tablet_service.cc:1467] Processing CreateTablet for tablet bb091eaf92de4f9bbf110f489a9f21bd (DEFAULT_TABLE table=rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:26.835274 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb091eaf92de4f9bbf110f489a9f21bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:26.835511 24995 tablet_bootstrap.cc:654] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.836375 25018 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:26.836462 25018 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240501 14:10:26.836472 24994 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:26.836541 24994 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:26.836817 25018 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.836933 25018 raft_consensus.cc:374] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.836966 25018 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.836928 24994 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.837033 24994 raft_consensus.cc:374] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.837064 24994 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.837061 25018 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.837175 24994 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.837320 25018 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:26.837427 24994 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:26.837477 25018 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:26.837582 24994 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:26.838222 25018 tablet_bootstrap.cc:654] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.838389 24994 tablet_bootstrap.cc:654] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.839190 25006 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:26.839265 25006 ts_tablet_manager.cc:1367] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20240501 14:10:26.839416 24996 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:26.839465 24998 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:26.839498 24996 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 14:10:26.839525 24998 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 14:10:26.839681 25006 raft_consensus.cc:348] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.839787 25006 raft_consensus.cc:374] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.839818 25006 raft_consensus.cc:729] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.839880 24996 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.839880 24998 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.839926 25006 consensus_queue.cc:260] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.839988 24996 raft_consensus.cc:374] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.839989 24998 raft_consensus.cc:374] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.840021 24996 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.840023 24998 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.840129 24998 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.840129 24996 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.840216 25006 ts_tablet_manager.cc:1398] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:26.840268 24993 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:26.840334 24993 ts_tablet_manager.cc:1367] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20240501 14:10:26.840375 24996 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.840375 24998 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:26.840379 25006 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:26.840520 24996 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:26.840520 24998 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:26.840725 24993 raft_consensus.cc:348] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.840835 24993 raft_consensus.cc:374] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.840865 24993 raft_consensus.cc:729] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.840981 24993 consensus_queue.cc:260] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.841168 25006 tablet_bootstrap.cc:654] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.841230 24993 ts_tablet_manager.cc:1398] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.841297 24996 tablet_bootstrap.cc:654] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.841297 24998 tablet_bootstrap.cc:654] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.841374 24993 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:26.842192 24993 tablet_bootstrap.cc:654] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.843166 25006 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:26.843240 25006 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 14:10:26.843286 24994 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:26.843365 24994 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240501 14:10:26.843660 25006 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.843767 25006 raft_consensus.cc:374] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.843722 24994 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.843796 25006 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.843812 24994 raft_consensus.cc:374] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.843839 24994 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.843924 25006 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.844168 25006 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.844131 24994 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.844363 25018 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:26.844444 25018 ts_tablet_manager.cc:1367] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240501 14:10:26.844626 24994 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.844821 24994 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:26.844852 25015 raft_consensus.cc:482] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:26.844873 25018 raft_consensus.cc:348] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.845000 25018 raft_consensus.cc:374] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.844934 25015 raft_consensus.cc:504] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.845041 25018 raft_consensus.cc:729] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.845149 25018 consensus_queue.cc:260] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.845413 25018 ts_tablet_manager.cc:1398] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.845579 25018 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:26.846047 24994 tablet_bootstrap.cc:654] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.846418 25018 tablet_bootstrap.cc:654] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.846545 25015 leader_election.cc:288] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:26.847136 24993 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:26.847214 24994 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:26.847241 24993 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:10:26.847281 24994 ts_tablet_manager.cc:1367] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:26.847679 24993 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.847679 24994 raft_consensus.cc:348] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.847784 24994 raft_consensus.cc:374] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.847784 24993 raft_consensus.cc:374] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.847813 24993 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.847813 24994 raft_consensus.cc:729] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.847945 24993 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.847932 24994 consensus_queue.cc:260] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.848201 24994 ts_tablet_manager.cc:1398] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.848250 24993 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:26.848937 24996 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:26.849020 24996 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240501 14:10:26.849332 25018 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:26.849398 25018 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:10:26.849424 24996 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.849572 24996 raft_consensus.cc:374] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.849611 24996 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.849726 24996 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.849774 25018 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.849877 25018 raft_consensus.cc:374] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.849908 25018 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.849999 24996 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.850010 25018 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.850154 24996 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:26.850252 25018 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.850996 24996 tablet_bootstrap.cc:654] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.851840 24995 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:26.851923 24995 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240501 14:10:26.852341 24995 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.852468 24995 raft_consensus.cc:374] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.852504 24995 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.852614 24995 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.852880 24995 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:26.853055 24995 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:26.853886 24995 tablet_bootstrap.cc:654] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:26.856019 24998 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:26.856107 24998 ts_tablet_manager.cc:1367] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.002s
I20240501 14:10:26.856081 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:26.856199 24390 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:26.856539 24998 raft_consensus.cc:348] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.856657 24998 raft_consensus.cc:374] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.856690 24998 raft_consensus.cc:729] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.856791 24998 consensus_queue.cc:260] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.856880 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:26.856969 24259 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:26.857038 24998 ts_tablet_manager.cc:1398] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:26.858527 24995 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:26.858604 24995 ts_tablet_manager.cc:1367] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:10:26.859191 24995 raft_consensus.cc:348] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.859297 24995 raft_consensus.cc:374] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.859352 24995 raft_consensus.cc:729] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.859468 24995 consensus_queue.cc:260] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.859716 24995 ts_tablet_manager.cc:1398] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:26.860231 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:26.860345 23864 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:26.860684 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:26.860709 24064 leader_election.cc:302] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:26.860788 23995 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:26.860929 25015 raft_consensus.cc:2781] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:26.860980 25015 raft_consensus.cc:482] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:26.861013 25015 raft_consensus.cc:3037] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.861972 25015 raft_consensus.cc:504] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.862514 24996 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:26.862521 25015 leader_election.cc:288] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [CANDIDATE]: Term 1 election: Requested vote from peers 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:26.862584 24996 ts_tablet_manager.cc:1367] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20240501 14:10:26.862967 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:26.863026 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:26.863070 23995 raft_consensus.cc:3037] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.863117 23864 raft_consensus.cc:3037] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.862960 24996 raft_consensus.cc:348] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.863571 24996 raft_consensus.cc:374] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:26.863612 24996 raft_consensus.cc:729] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:26.863708 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:26.863718 24996 consensus_queue.cc:260] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.863807 24523 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:26.863952 23995 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:26.863976 24996 ts_tablet_manager.cc:1398] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:26.864001 23864 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:26.864148 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:26.864253 24524 raft_consensus.cc:3037] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.864254 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:26.864343 24390 raft_consensus.cc:3037] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.865234 24390 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:26.865242 24524 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:26.865585 24061 leader_election.cc:302] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:26.865901 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:26.865943 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:26.866009 24259 raft_consensus.cc:3037] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.866040 23731 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:26.866137 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee07797b5c84ad8aa71d10c1ac24fe1" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:26.866235 23732 raft_consensus.cc:3037] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.866890 24259 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:26.867151 23732 raft_consensus.cc:2454] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:26.867386 25015 raft_consensus.cc:2781] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:26.867538 25015 raft_consensus.cc:686] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Becoming Leader. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Running, Role: LEADER
I20240501 14:10:26.867833 25015 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:26.870122 23590 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 reported cstate change: term changed from 0 to 1, leader changed from <none> to 166d7170d98944e88de2eccda9d30545 (127.6.153.196). New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:26.873613 25142 raft_consensus.cc:482] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:26.873694 25142 raft_consensus.cc:504] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.874258 25142 leader_election.cc:288] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.874343 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:26.874343 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:26.874442 23864 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.874464 23732 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.874519 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:26.874636 24259 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.874826 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:26.874814 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:26.874912 24127 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.874919 24390 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.875149 23931 leader_election.cc:302] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:26.875310 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:26.875361 25142 raft_consensus.cc:2781] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:26.875413 25142 raft_consensus.cc:482] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:26.875454 25142 raft_consensus.cc:3037] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.875439 24524 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:26.876133 25142 raft_consensus.cc:504] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.876610 25142 leader_election.cc:288] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 election: Requested vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.876724 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:26.876811 23864 raft_consensus.cc:3037] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.876792 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:26.876873 24259 raft_consensus.cc:3037] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.877153 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:26.877241 24127 raft_consensus.cc:3037] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.877606 23864 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.877606 24259 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.877805 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:26.877864 24127 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.877894 24390 raft_consensus.cc:3037] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.878160 23931 leader_election.cc:302] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:26.878306 25142 raft_consensus.cc:2781] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:26.878368 25142 raft_consensus.cc:686] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Becoming Leader. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Running, Role: LEADER
I20240501 14:10:26.878489 25142 consensus_queue.cc:237] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.878649 24390 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.878850 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:26.878940 23732 raft_consensus.cc:3037] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.879776 23732 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.880326 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb091eaf92de4f9bbf110f489a9f21bd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:26.880425 24524 raft_consensus.cc:3037] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.880640 23590 catalog_manager.cc:5484] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195). New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:26.881058 24524 raft_consensus.cc:2454] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:26.904732 25222 raft_consensus.cc:482] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:26.904839 25222 raft_consensus.cc:504] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.905642 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:26.905687 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:26.905754 23995 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.905809 23864 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.906116 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:26.906235 24390 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.906258 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:26.906363 24524 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.906335 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:26.906442 24127 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.905470 25222 leader_election.cc:288] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.906613 23667 leader_election.cc:302] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 40b69f74de5a447f86c1445f8e84850d, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:26.906801 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:26.906919 24259 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:26.906984 25222 raft_consensus.cc:2781] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:26.907032 25222 raft_consensus.cc:482] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:26.907063 25222 raft_consensus.cc:3037] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.908048 25222 raft_consensus.cc:504] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.908594 25222 leader_election.cc:288] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 election: Requested vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:26.908797 24390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:26.908788 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:26.908879 23864 raft_consensus.cc:3037] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.908879 24390 raft_consensus.cc:3037] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.909247 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:26.909333 24127 raft_consensus.cc:3037] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.909703 24390 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.909703 23864 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.910005 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:26.910106 24524 raft_consensus.cc:3037] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.910149 24127 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.910156 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:26.910243 23995 raft_consensus.cc:3037] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.910917 24524 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.910921 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fda65c1f5240ce9f886d57921a03f0" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:26.911000 23995 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.911016 24259 raft_consensus.cc:3037] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:26.911187 23668 leader_election.cc:302] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04; no voters:
I20240501 14:10:26.911435 25222 raft_consensus.cc:2781] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:26.911520 25222 raft_consensus.cc:686] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Becoming Leader. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Running, Role: LEADER
I20240501 14:10:26.911666 25222 consensus_queue.cc:237] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:26.911835 24259 raft_consensus.cc:2454] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:26.913645 23590 catalog_manager.cc:5484] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193). New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
W20240501 14:10:27.028012 25277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:27.028203 25277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:27.031065 25277 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 14:10:27.100677 24259 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.101223 24524 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.101559 24127 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.101586 23732 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.101853 24390 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.102197 24392 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.102223 23864 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.102675 23995 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.102643 25273 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.102749 24259 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.102864 25219 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.103668 24392 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.104027 25142 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.104082 24523 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.104182 24524 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.104192 23864 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.104542 25128 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.104672 25219 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.105002 25128 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.105042 25037 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.105298 23995 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.103981 25303 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240501 14:10:27.105724 25273 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.106206 25221 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.106258 25273 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.106503 25221 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.107679 23864 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.108053 24259 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.111162 25142 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.111622 25306 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.111743 25015 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.118021 24127 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.119171 23732 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.120085 25219 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.120430 25128 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.120704 25128 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 195.254 ms
time per row: 0.0976269 ms
I20240501 14:10:27.263499 23584 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37374:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:27.263962 23584 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_3 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 14:10:27.268944 23712 tablet_service.cc:1467] Processing CreateTablet for tablet 207a860a62e34c619ecec37f1eacaacd (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.269086 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 207a860a62e34c619ecec37f1eacaacd (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.269237 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 207a860a62e34c619ecec37f1eacaacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.269320 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 207a860a62e34c619ecec37f1eacaacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.270843 24370 tablet_service.cc:1467] Processing CreateTablet for tablet 207a860a62e34c619ecec37f1eacaacd (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.271140 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 207a860a62e34c619ecec37f1eacaacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.271256 24994 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:27.271395 24995 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:27.271782 23974 tablet_service.cc:1467] Processing CreateTablet for tablet 207a860a62e34c619ecec37f1eacaacd (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.272029 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 207a860a62e34c619ecec37f1eacaacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.272243 24995 tablet_bootstrap.cc:654] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.272243 24994 tablet_bootstrap.cc:654] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.271149 24372 tablet_service.cc:1467] Processing CreateTablet for tablet 158b5efe75ce450abd20149d89157edc (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.273221 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158b5efe75ce450abd20149d89157edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.273178 24239 tablet_service.cc:1467] Processing CreateTablet for tablet 158b5efe75ce450abd20149d89157edc (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.273291 24993 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:27.273420 24239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158b5efe75ce450abd20149d89157edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.274051 24993 tablet_bootstrap.cc:654] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.270848 24371 tablet_service.cc:1467] Processing CreateTablet for tablet fccb18388fb244bcb1dfd85908c0c118 (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.274780 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.274740 23711 tablet_service.cc:1467] Processing CreateTablet for tablet 158b5efe75ce450abd20149d89157edc (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.274940 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158b5efe75ce450abd20149d89157edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.275409 24998 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:27.276160 24998 tablet_bootstrap.cc:654] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.276603 23844 tablet_service.cc:1467] Processing CreateTablet for tablet 158b5efe75ce450abd20149d89157edc (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.276818 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158b5efe75ce450abd20149d89157edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.276940 24240 tablet_service.cc:1467] Processing CreateTablet for tablet fccb18388fb244bcb1dfd85908c0c118 (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.277132 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.277876 23843 tablet_service.cc:1467] Processing CreateTablet for tablet fccb18388fb244bcb1dfd85908c0c118 (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.278033 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.278829 24996 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:27.279493 24996 tablet_bootstrap.cc:654] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.279650 23976 tablet_service.cc:1467] Processing CreateTablet for tablet 158b5efe75ce450abd20149d89157edc (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.279650 23974 tablet_service.cc:1467] Processing CreateTablet for tablet fccb18388fb244bcb1dfd85908c0c118 (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.279904 23976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158b5efe75ce450abd20149d89157edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.280063 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.273178 24238 tablet_service.cc:1467] Processing CreateTablet for tablet 207a860a62e34c619ecec37f1eacaacd (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.280267 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 207a860a62e34c619ecec37f1eacaacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.280838 24106 tablet_service.cc:1467] Processing CreateTablet for tablet 207a860a62e34c619ecec37f1eacaacd (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.281078 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 207a860a62e34c619ecec37f1eacaacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.282011 24108 tablet_service.cc:1467] Processing CreateTablet for tablet 158b5efe75ce450abd20149d89157edc (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.282222 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158b5efe75ce450abd20149d89157edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.283030 25006 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:27.283085 24107 tablet_service.cc:1467] Processing CreateTablet for tablet fccb18388fb244bcb1dfd85908c0c118 (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.283317 24998 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:27.283305 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.283411 24998 ts_tablet_manager.cc:1367] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240501 14:10:27.283790 24998 raft_consensus.cc:348] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.283893 24998 raft_consensus.cc:374] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.283921 24998 raft_consensus.cc:729] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.284018 24998 consensus_queue.cc:260] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.284103 25006 tablet_bootstrap.cc:654] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.284294 24998 ts_tablet_manager.cc:1398] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.284451 24998 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:27.284435 24504 tablet_service.cc:1467] Processing CreateTablet for tablet 207a860a62e34c619ecec37f1eacaacd (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.284688 24504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 207a860a62e34c619ecec37f1eacaacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.285140 24998 tablet_bootstrap.cc:654] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.285255 24993 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:27.285322 24993 ts_tablet_manager.cc:1367] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20240501 14:10:27.285722 24993 raft_consensus.cc:348] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.285836 24993 raft_consensus.cc:374] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.285866 24993 raft_consensus.cc:729] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.285980 24993 consensus_queue.cc:260] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.286232 24993 ts_tablet_manager.cc:1398] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:27.286329 24996 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:27.286401 24996 ts_tablet_manager.cc:1367] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240501 14:10:27.286401 24993 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:27.286778 24502 tablet_service.cc:1467] Processing CreateTablet for tablet fccb18388fb244bcb1dfd85908c0c118 (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.286800 24996 raft_consensus.cc:348] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.286903 24996 raft_consensus.cc:374] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.286934 24996 raft_consensus.cc:729] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.286989 24502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.287070 24996 consensus_queue.cc:260] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.287178 24993 tablet_bootstrap.cc:654] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.287324 24996 ts_tablet_manager.cc:1398] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.287497 24996 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:27.288419 24996 tablet_bootstrap.cc:654] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.288775 24995 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:27.288794 24503 tablet_service.cc:1467] Processing CreateTablet for tablet 158b5efe75ce450abd20149d89157edc (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.288856 24995 ts_tablet_manager.cc:1367] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240501 14:10:27.288995 24503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158b5efe75ce450abd20149d89157edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.289268 24995 raft_consensus.cc:348] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.289419 24995 raft_consensus.cc:374] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.289461 24995 raft_consensus.cc:729] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.289580 24995 consensus_queue.cc:260] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.289852 24995 ts_tablet_manager.cc:1398] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.290002 24995 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:27.290634 24995 tablet_bootstrap.cc:654] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.290797 25018 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:27.291596 25018 tablet_bootstrap.cc:654] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.291579 24995 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:27.291771 24995 ts_tablet_manager.cc:1367] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 14:10:27.291934 24998 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:27.292008 24998 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 14:10:27.292160 24995 raft_consensus.cc:348] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.292254 24995 raft_consensus.cc:374] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.292291 24995 raft_consensus.cc:729] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.292376 24998 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.292482 24998 raft_consensus.cc:374] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.292418 24995 consensus_queue.cc:260] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.292515 24998 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.292626 24998 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.292852 24995 ts_tablet_manager.cc:1398] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:27.293473 23712 tablet_service.cc:1467] Processing CreateTablet for tablet fccb18388fb244bcb1dfd85908c0c118 (DEFAULT_TABLE table=rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.293682 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.295359 24998 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240501 14:10:27.295933 24998 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:27.296711 24994 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:27.296790 24994 ts_tablet_manager.cc:1367] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240501 14:10:27.297174 24994 raft_consensus.cc:348] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.297284 24994 raft_consensus.cc:374] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.297309 24994 raft_consensus.cc:729] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.297375 24998 tablet_bootstrap.cc:654] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.297410 24994 consensus_queue.cc:260] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.297649 24994 ts_tablet_manager.cc:1398] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.297785 24994 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:27.298668 25006 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:27.298748 25006 ts_tablet_manager.cc:1367] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240501 14:10:27.299124 25006 raft_consensus.cc:348] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.299232 25006 raft_consensus.cc:374] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.299257 25006 raft_consensus.cc:729] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.299396 25006 consensus_queue.cc:260] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.299657 25006 ts_tablet_manager.cc:1398] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:27.299804 25006 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:27.300585 25006 tablet_bootstrap.cc:654] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.304128 24996 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:27.304208 24996 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240501 14:10:27.304509 25006 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:27.304584 25006 ts_tablet_manager.cc:1367] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240501 14:10:27.304520 24996 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.304620 24996 raft_consensus.cc:374] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.304654 24996 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.304770 24996 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.304931 25006 raft_consensus.cc:348] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.305012 24996 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.305032 25006 raft_consensus.cc:374] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.305060 25006 raft_consensus.cc:729] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.305183 24996 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:27.305161 25006 consensus_queue.cc:260] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.303303 24994 tablet_bootstrap.cc:654] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.305912 24996 tablet_bootstrap.cc:654] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.305923 25006 ts_tablet_manager.cc:1398] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.306072 25006 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:27.308089 25006 tablet_bootstrap.cc:654] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.308573 24995 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:27.309379 24995 tablet_bootstrap.cc:654] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.310382 25018 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:27.310472 25018 ts_tablet_manager.cc:1367] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240501 14:10:27.310608 24993 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:27.310688 24993 ts_tablet_manager.cc:1367] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.001s
I20240501 14:10:27.310899 25018 raft_consensus.cc:348] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.311043 25018 raft_consensus.cc:374] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.311084 25018 raft_consensus.cc:729] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.311075 24993 raft_consensus.cc:348] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.311187 24993 raft_consensus.cc:374] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.311224 24993 raft_consensus.cc:729] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.311208 25018 consensus_queue.cc:260] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.311501 25018 ts_tablet_manager.cc:1398] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.311481 24993 consensus_queue.cc:260] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.311651 25018 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:27.311776 24993 ts_tablet_manager.cc:1398] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:27.311933 24993 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:27.312464 25018 tablet_bootstrap.cc:654] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.312759 24993 tablet_bootstrap.cc:654] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.313493 25018 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:27.313572 25018 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:27.313961 25018 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.314069 25018 raft_consensus.cc:374] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.314100 25018 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.314291 25018 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.314630 25018 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.314785 25018 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:27.315018 24994 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:27.315111 24994 ts_tablet_manager.cc:1367] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240501 14:10:27.315619 25018 tablet_bootstrap.cc:654] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.315611 24994 raft_consensus.cc:348] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.315721 24994 raft_consensus.cc:374] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.315747 24994 raft_consensus.cc:729] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.315862 24994 consensus_queue.cc:260] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.316195 24994 ts_tablet_manager.cc:1398] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.316488 24994 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:27.318611 24994 tablet_bootstrap.cc:654] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.319478 24994 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:27.319540 24994 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:10:27.319866 24994 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.319969 24994 raft_consensus.cc:374] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.319998 24994 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.320108 24994 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.320362 24994 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.327644 24993 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:27.327730 24993 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240501 14:10:27.327765 24998 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:27.327848 24998 ts_tablet_manager.cc:1367] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.001s
I20240501 14:10:27.328107 24993 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.328186 24993 raft_consensus.cc:374] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.328204 24993 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.328297 24993 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.328266 24998 raft_consensus.cc:348] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.328379 24998 raft_consensus.cc:374] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.328410 24998 raft_consensus.cc:729] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.328521 24993 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.328533 24998 consensus_queue.cc:260] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.328905 24998 ts_tablet_manager.cc:1398] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.329350 25006 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:27.329434 25006 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20240501 14:10:27.329851 25006 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.329964 25006 raft_consensus.cc:374] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.329998 25006 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.330133 25006 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.330524 25006 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.331732 24995 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:27.331921 24995 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.002s
I20240501 14:10:27.332312 24995 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.332410 24995 raft_consensus.cc:374] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.332437 24995 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.332549 24995 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.332806 24995 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.335778 25330 raft_consensus.cc:482] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:27.335906 25330 raft_consensus.cc:504] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.336611 25330 leader_election.cc:288] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:27.336750 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:27.336882 23863 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.336891 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:27.337013 23731 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.337086 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:27.337194 24127 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.337582 23930 leader_election.cc:302] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:27.337683 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:27.337738 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:27.337788 24524 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.337777 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:27.337850 24259 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.337867 24392 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.338101 25087 raft_consensus.cc:482] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:27.338186 25087 raft_consensus.cc:504] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.338778 25087 leader_election.cc:288] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:27.338933 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:27.339035 23863 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.339123 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:27.339205 25330 raft_consensus.cc:2781] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:27.339218 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:27.339222 23996 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.339275 25330 raft_consensus.cc:482] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:27.339311 25330 raft_consensus.cc:3037] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.339311 24259 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.339516 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:27.339603 23731 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.339701 25018 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:27.339735 24996 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:27.339777 25018 ts_tablet_manager.cc:1367] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240501 14:10:27.339810 24996 ts_tablet_manager.cc:1367] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.000s
I20240501 14:10:27.339985 24326 leader_election.cc:302] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:27.340093 25018 raft_consensus.cc:348] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.340126 25087 raft_consensus.cc:2781] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:27.340169 25018 raft_consensus.cc:374] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.340179 25087 raft_consensus.cc:482] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:27.340188 25018 raft_consensus.cc:729] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.340214 25087 raft_consensus.cc:3037] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.340224 24996 raft_consensus.cc:348] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.340319 24996 raft_consensus.cc:374] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.340286 25018 consensus_queue.cc:260] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.340349 24996 raft_consensus.cc:729] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.340441 24996 consensus_queue.cc:260] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.340543 25018 ts_tablet_manager.cc:1398] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.340715 24996 ts_tablet_manager.cc:1398] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.341523 25087 raft_consensus.cc:504] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.341737 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:27.341825 24127 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.342094 25087 leader_election.cc:288] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Requested vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:27.342247 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:27.342332 23863 raft_consensus.cc:3037] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.342314 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:27.342392 23996 raft_consensus.cc:3037] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.342391 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:27.342490 24524 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.343238 23996 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.343238 23863 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.343379 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:27.343477 24127 raft_consensus.cc:3037] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.343937 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:27.344053 23731 raft_consensus.cc:3037] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.344424 24127 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.344686 24327 leader_election.cc:302] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:27.344986 23731 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.345103 25090 raft_consensus.cc:2781] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:27.345206 25090 raft_consensus.cc:686] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Becoming Leader. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Running, Role: LEADER
I20240501 14:10:27.343740 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:27.345399 24259 raft_consensus.cc:3037] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.345342 25090 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.345425 25330 raft_consensus.cc:504] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.345976 25330 leader_election.cc:288] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 election: Requested vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:27.346246 24259 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.346298 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:27.346382 24127 raft_consensus.cc:3037] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.347087 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:27.347147 24127 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.347244 23863 raft_consensus.cc:3037] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.347249 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:27.347385 24392 raft_consensus.cc:3037] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.348062 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fccb18388fb244bcb1dfd85908c0c118" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:27.348083 23863 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.348155 24523 raft_consensus.cc:3037] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.348217 24392 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.349028 24523 raft_consensus.cc:2454] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.349017 23584 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198). New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:27.349472 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:27.349556 24524 raft_consensus.cc:3037] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.350334 24524 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.351612 23929 leader_election.cc:302] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:27.351760 25330 raft_consensus.cc:2781] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:27.351894 25330 raft_consensus.cc:686] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Becoming Leader. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Running, Role: LEADER
I20240501 14:10:27.352280 25330 consensus_queue.cc:237] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.352368 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:27.352455 24259 raft_consensus.cc:3037] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.353233 24259 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.353611 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "207a860a62e34c619ecec37f1eacaacd" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:27.353720 23731 raft_consensus.cc:3037] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.354542 23731 raft_consensus.cc:2454] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.354769 23584 catalog_manager.cc:5484] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195). New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:27.432898 25226 raft_consensus.cc:482] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:27.433015 25226 raft_consensus.cc:504] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.433600 25226 leader_election.cc:288] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:27.433804 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:27.433804 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:27.433935 24127 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.433935 24524 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.434020 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:27.434154 23863 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.434257 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:27.434391 23731 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.434402 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:27.434432 24196 leader_election.cc:302] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 40b69f74de5a447f86c1445f8e84850d, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:27.434530 23996 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.434602 25226 raft_consensus.cc:2781] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:27.434576 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:27.434641 25226 raft_consensus.cc:482] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:27.434671 25226 raft_consensus.cc:3037] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.434692 24392 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.435693 25226 raft_consensus.cc:504] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.436259 25226 leader_election.cc:288] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 election: Requested vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:27.436453 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:27.436501 23863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:27.436511 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:27.436571 24127 raft_consensus.cc:3037] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.436591 23863 raft_consensus.cc:3037] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.436595 24524 raft_consensus.cc:3037] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.437237 23863 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.437430 24524 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.437430 24127 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.437664 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:27.437742 24195 leader_election.cc:302] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 40b69f74de5a447f86c1445f8e84850d, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:27.437796 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:27.437898 24392 raft_consensus.cc:3037] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.437901 25226 raft_consensus.cc:2781] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:27.437891 23996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158b5efe75ce450abd20149d89157edc" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:27.437904 23731 raft_consensus.cc:3037] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.437981 25226 raft_consensus.cc:686] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Becoming Leader. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Running, Role: LEADER
I20240501 14:10:27.438011 23996 raft_consensus.cc:3037] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.438131 25226 consensus_queue.cc:237] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.438755 24392 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.438876 23996 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.439030 23731 raft_consensus.cc:2454] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.440251 23584 catalog_manager.cc:5484] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197). New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
W20240501 14:10:27.568089 25356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:27.568346 25356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:27.571393 25356 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 14:10:27.662802 24259 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.662847 24127 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.663214 25330 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.663357 25142 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.664176 24392 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.664191 24524 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.664572 25329 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.664978 25142 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.666285 24258 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.666532 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.666787 24524 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.667135 25090 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.667259 23863 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.667790 25090 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.669721 24128 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.670045 25087 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.670320 25087 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.672541 25090 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.675931 23996 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.676899 25087 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.680474 23864 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.680826 23732 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.686664 23732 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.687296 24127 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.687477 24522 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.687908 25329 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.688350 25225 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.690186 25226 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.690513 25226 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.692771 24390 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.693122 25225 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.695417 23864 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.696225 23996 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:27.696941 25142 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.698001 25226 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.698257 25226 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:27.739732 25109 mvcc.cc:205] Tried to move back new op lower bound from 7022889482997518336 to 7022889481989242880. Current Snapshot: MvccSnapshot[applied={T|T < 7022889482902274048}]
I20240501 14:10:27.740123 25085 mvcc.cc:205] Tried to move back new op lower bound from 7022889482997518336 to 7022889481989242880. Current Snapshot: MvccSnapshot[applied={T|T < 7022889482902274048}]
I20240501 14:10:27.740550 25091 mvcc.cc:205] Tried to move back new op lower bound from 7022889482915299328 to 7022889481610604544. Current Snapshot: MvccSnapshot[applied={T|T < 7022889482915299328}]
I20240501 14:10:27.742383 25152 mvcc.cc:205] Tried to move back new op lower bound from 7022889482903511040 to 7022889481640456192. Current Snapshot: MvccSnapshot[applied={T|T < 7022889482903511040}]
I20240501 14:10:27.747279 25107 mvcc.cc:205] Tried to move back new op lower bound from 7022889482989604864 to 7022889481640456192. Current Snapshot: MvccSnapshot[applied={T|T < 7022889482903511040}]
I20240501 14:10:27.775568 25110 mvcc.cc:205] Tried to move back new op lower bound from 7022889482997518336 to 7022889481989242880. Current Snapshot: MvccSnapshot[applied={T|T < 7022889482997518336}]
INSERT report
rows total: 2000
time total: 198.136 ms
time per row: 0.0990679 ms
I20240501 14:10:27.812565 23584 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37374:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:27.813082 23584 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_4 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 14:10:27.818147 23843 tablet_service.cc:1467] Processing CreateTablet for tablet be60ec7c4b0a46c28e0b23b6facae47e (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.818199 23712 tablet_service.cc:1467] Processing CreateTablet for tablet be60ec7c4b0a46c28e0b23b6facae47e (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.818423 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be60ec7c4b0a46c28e0b23b6facae47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.818425 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be60ec7c4b0a46c28e0b23b6facae47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.820396 23976 tablet_service.cc:1467] Processing CreateTablet for tablet be60ec7c4b0a46c28e0b23b6facae47e (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.820674 23976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be60ec7c4b0a46c28e0b23b6facae47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.820746 23975 tablet_service.cc:1467] Processing CreateTablet for tablet 84da4677d89b42fe8c2418cd156c75ae (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.820930 24995 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:27.820973 23975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.821719 24995 tablet_bootstrap.cc:654] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.822779 24994 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:27.822773 24996 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:27.823482 24994 tablet_bootstrap.cc:654] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.823616 24239 tablet_service.cc:1467] Processing CreateTablet for tablet be60ec7c4b0a46c28e0b23b6facae47e (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.823851 24239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be60ec7c4b0a46c28e0b23b6facae47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.823966 24107 tablet_service.cc:1467] Processing CreateTablet for tablet be60ec7c4b0a46c28e0b23b6facae47e (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.824215 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be60ec7c4b0a46c28e0b23b6facae47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.824414 23844 tablet_service.cc:1467] Processing CreateTablet for tablet 84da4677d89b42fe8c2418cd156c75ae (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.824611 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.825891 24238 tablet_service.cc:1467] Processing CreateTablet for tablet 3377f84deb6249668991e5a3f6d63876 (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.826092 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.826184 24106 tablet_service.cc:1467] Processing CreateTablet for tablet 84da4677d89b42fe8c2418cd156c75ae (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.826377 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.820771 23974 tablet_service.cc:1467] Processing CreateTablet for tablet 3377f84deb6249668991e5a3f6d63876 (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.826603 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.827919 24240 tablet_service.cc:1467] Processing CreateTablet for tablet 84da4677d89b42fe8c2418cd156c75ae (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.828114 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.828114 25006 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:27.828809 25006 tablet_bootstrap.cc:654] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.829473 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 3377f84deb6249668991e5a3f6d63876 (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.829704 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.829912 24108 tablet_service.cc:1467] Processing CreateTablet for tablet 3377f84deb6249668991e5a3f6d63876 (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.830097 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.831599 24995 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:27.831604 24998 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:27.831682 24995 ts_tablet_manager.cc:1367] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20240501 14:10:27.832005 24995 raft_consensus.cc:348] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.832089 24995 raft_consensus.cc:374] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.832108 24995 raft_consensus.cc:729] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.832197 24995 consensus_queue.cc:260] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.832262 24998 tablet_bootstrap.cc:654] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.832403 24995 ts_tablet_manager.cc:1398] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.832871 24504 tablet_service.cc:1467] Processing CreateTablet for tablet 84da4677d89b42fe8c2418cd156c75ae (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.833153 24504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.833273 24998 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:27.833354 24998 ts_tablet_manager.cc:1367] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:27.833657 24998 raft_consensus.cc:348] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.833756 24998 raft_consensus.cc:374] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.833787 24998 raft_consensus.cc:729] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.833884 24998 consensus_queue.cc:260] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.834116 24998 ts_tablet_manager.cc:1398] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.834265 24998 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:27.835000 25018 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:27.835125 24998 tablet_bootstrap.cc:654] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.835224 24502 tablet_service.cc:1467] Processing CreateTablet for tablet 3377f84deb6249668991e5a3f6d63876 (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.835460 24502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.835914 25018 tablet_bootstrap.cc:654] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.836086 24998 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:27.836158 24998 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:27.836539 24998 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.836655 24998 raft_consensus.cc:374] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.836688 24998 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.836778 24998 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.836971 25018 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:27.837055 25018 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 14:10:27.837069 24998 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.837210 24998 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:27.837369 24503 tablet_service.cc:1467] Processing CreateTablet for tablet be60ec7c4b0a46c28e0b23b6facae47e (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.837446 25018 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.837567 25018 raft_consensus.cc:374] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.837589 24503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be60ec7c4b0a46c28e0b23b6facae47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.837602 25018 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.837987 24998 tablet_bootstrap.cc:654] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.838112 25018 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.838400 25018 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:27.838572 25018 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:27.839236 24998 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:27.839519 25018 tablet_bootstrap.cc:654] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.839854 23712 tablet_service.cc:1467] Processing CreateTablet for tablet 3377f84deb6249668991e5a3f6d63876 (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.840077 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.840055 24371 tablet_service.cc:1467] Processing CreateTablet for tablet be60ec7c4b0a46c28e0b23b6facae47e (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.840304 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be60ec7c4b0a46c28e0b23b6facae47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.840305 24996 tablet_bootstrap.cc:654] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.840325 23711 tablet_service.cc:1467] Processing CreateTablet for tablet 84da4677d89b42fe8c2418cd156c75ae (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.840512 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.841754 25006 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:27.841833 25006 ts_tablet_manager.cc:1367] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240501 14:10:27.842254 25006 raft_consensus.cc:348] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.842389 25006 raft_consensus.cc:374] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.842427 25006 raft_consensus.cc:729] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.842617 24998 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 14:10:27.842571 25006 consensus_queue.cc:260] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.842839 25006 ts_tablet_manager.cc:1398] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.842923 24994 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:27.843001 25006 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:27.843024 24994 ts_tablet_manager.cc:1367] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20240501 14:10:27.843062 24998 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.843173 24998 raft_consensus.cc:374] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.843206 24998 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.843302 24998 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.843469 24994 raft_consensus.cc:348] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.843578 24994 raft_consensus.cc:374] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.843609 24994 raft_consensus.cc:729] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.843658 24998 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.843721 24994 consensus_queue.cc:260] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.843850 24372 tablet_service.cc:1467] Processing CreateTablet for tablet 3377f84deb6249668991e5a3f6d63876 (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.843971 25006 tablet_bootstrap.cc:654] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.843994 24994 ts_tablet_manager.cc:1398] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.844086 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.844139 24994 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:27.844812 24370 tablet_service.cc:1467] Processing CreateTablet for tablet 84da4677d89b42fe8c2418cd156c75ae (DEFAULT_TABLE table=rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:27.844866 24994 tablet_bootstrap.cc:654] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.845017 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:27.845064 25006 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:27.845131 25006 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:27.845528 25006 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.845623 25006 raft_consensus.cc:374] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.845652 25006 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.845775 25006 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.846037 25006 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.846196 25006 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:27.848642 24995 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:27.849450 24995 tablet_bootstrap.cc:654] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.849777 25006 tablet_bootstrap.cc:654] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.851619 25018 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:27.851708 25018 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240501 14:10:27.851707 25449 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:27.851859 24996 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:27.851953 24996 ts_tablet_manager.cc:1367] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20240501 14:10:27.852126 25018 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.852243 25018 raft_consensus.cc:374] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.852281 25018 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.852399 25018 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.852382 24996 raft_consensus.cc:348] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.852516 24996 raft_consensus.cc:374] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.852550 24996 raft_consensus.cc:729] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.852690 25018 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:27.852658 24996 consensus_queue.cc:260] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.852838 25018 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:27.852921 24996 ts_tablet_manager.cc:1398] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.853013 25449 tablet_bootstrap.cc:654] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.853107 24996 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:27.853533 25018 tablet_bootstrap.cc:654] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.853925 24996 tablet_bootstrap.cc:654] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.854851 24996 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:27.854923 24996 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:27.855360 24996 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.855468 24996 raft_consensus.cc:374] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.855495 24996 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.854593 25449 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:27.855600 24996 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.855839 24996 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.855902 25449 ts_tablet_manager.cc:1367] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:10:27.855983 24996 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:27.856351 25449 raft_consensus.cc:348] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.856480 25449 raft_consensus.cc:374] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.856518 25449 raft_consensus.cc:729] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.856628 25449 consensus_queue.cc:260] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.856770 24996 tablet_bootstrap.cc:654] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.856912 25449 ts_tablet_manager.cc:1398] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.857072 25449 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:27.857395 24995 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:27.857474 24995 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:10:27.857826 25449 tablet_bootstrap.cc:654] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.857867 24995 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.857995 24995 raft_consensus.cc:374] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.858027 24995 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.858136 24995 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.858403 24995 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.858561 24995 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:27.859520 24995 tablet_bootstrap.cc:654] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.861011 24995 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:27.861089 24995 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:10:27.861451 24995 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.861557 24995 raft_consensus.cc:374] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.861588 24995 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.861701 24995 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.862040 24995 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.862679 25018 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:27.862764 25018 ts_tablet_manager.cc:1367] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240501 14:10:27.863135 25018 raft_consensus.cc:348] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.863256 25018 raft_consensus.cc:374] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.863293 25018 raft_consensus.cc:729] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.863431 25018 consensus_queue.cc:260] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.863673 25018 ts_tablet_manager.cc:1398] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.863902 25006 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:27.863984 25006 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240501 14:10:27.864331 25006 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.864426 24996 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:27.864436 25006 raft_consensus.cc:374] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.864468 25006 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.864498 24996 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:10:27.864576 25006 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.864853 25006 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.865357 24996 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.865453 24996 raft_consensus.cc:374] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.865482 24996 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.865633 25449 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:27.865614 24996 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.865720 25449 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:10:27.866173 24996 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:27.866127 25449 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.866228 25449 raft_consensus.cc:374] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.866261 25449 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.866367 25449 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.866734 25449 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.866897 25449 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:27.867843 25449 tablet_bootstrap.cc:654] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.868808 25449 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:27.868889 25449 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:27.869278 25449 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.869387 25449 raft_consensus.cc:374] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.869429 25449 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.869427 25329 raft_consensus.cc:482] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:27.869532 25329 raft_consensus.cc:504] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.869540 25449 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.869777 25449 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.870134 25329 leader_election.cc:288] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:27.870401 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:27.870421 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:27.870451 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:27.870515 24392 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.870507 24127 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.870536 24523 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.870836 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:27.870939 23732 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.870949 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:27.871016 23931 leader_election.cc:302] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 40b69f74de5a447f86c1445f8e84850d, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:27.871062 24259 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.871203 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:27.871313 23862 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 0.
I20240501 14:10:27.871495 25329 raft_consensus.cc:2781] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:27.871542 25329 raft_consensus.cc:482] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:27.871575 25329 raft_consensus.cc:3037] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.871627 24994 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:27.871691 24994 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240501 14:10:27.872103 24994 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.872201 24994 raft_consensus.cc:374] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.872227 24994 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.872344 24994 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.872620 24994 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:27.872853 24994 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:27.872807 25329 raft_consensus.cc:504] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.873361 25329 leader_election.cc:288] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 election: Requested vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:27.873457 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:27.873543 23732 raft_consensus.cc:3037] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.873657 24994 tablet_bootstrap.cc:654] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:27.874073 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:27.874158 23862 raft_consensus.cc:3037] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.874228 23732 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.874565 24994 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:27.874536 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:27.874539 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:27.874629 24994 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:27.874625 24392 raft_consensus.cc:3037] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.874625 24259 raft_consensus.cc:3037] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.874990 24994 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.875079 23862 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.875097 24994 raft_consensus.cc:374] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:27.875126 24994 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:27.875222 24994 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.875416 24392 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.875456 24259 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.875488 24994 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:27.875790 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:27.875826 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be60ec7c4b0a46c28e0b23b6facae47e" candidate_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:27.875885 24127 raft_consensus.cc:3037] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.875917 24523 raft_consensus.cc:3037] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.875963 23930 leader_election.cc:302] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:27.876775 24523 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.876777 24127 raft_consensus.cc:2454] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebe0a098b0f4ed4861a5605717fecbd in term 1.
I20240501 14:10:27.877584 25329 raft_consensus.cc:2781] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:27.877669 25329 raft_consensus.cc:686] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Becoming Leader. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Running, Role: LEADER
I20240501 14:10:27.877800 25329 consensus_queue.cc:237] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:27.879619 23584 catalog_manager.cc:5484] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195). New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:27.898537 25402 raft_consensus.cc:482] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:27.898661 25402 raft_consensus.cc:504] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.899255 25402 leader_election.cc:288] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:27.899601 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:27.899703 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:27.899734 24127 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.899812 24392 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.899812 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:27.899909 23862 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.899896 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:27.900007 23995 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.900013 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:27.900120 24523 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.900234 24196 leader_election.cc:302] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:27.900332 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:27.900388 25402 raft_consensus.cc:2781] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:27.900425 25402 raft_consensus.cc:482] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:27.900450 23732 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 0.
I20240501 14:10:27.900493 25402 raft_consensus.cc:3037] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.901345 25402 raft_consensus.cc:504] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.901934 25402 leader_election.cc:288] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 election: Requested vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:27.901997 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:27.902109 24392 raft_consensus.cc:3037] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.902163 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:27.902173 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:27.902246 24523 raft_consensus.cc:3037] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.902261 23732 raft_consensus.cc:3037] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.902321 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:27.902402 23995 raft_consensus.cc:3037] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.902961 24392 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.903015 24523 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.903103 23732 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.903162 23995 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.903414 24194 leader_election.cc:302] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 40b69f74de5a447f86c1445f8e84850d, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:27.903446 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:27.903533 24127 raft_consensus.cc:3037] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.903573 25402 raft_consensus.cc:2781] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:27.903573 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3377f84deb6249668991e5a3f6d63876" candidate_uuid: "dbeb6a07fa7a44019ae00c169c414529" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:27.903641 25402 raft_consensus.cc:686] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Becoming Leader. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Running, Role: LEADER
I20240501 14:10:27.903684 23862 raft_consensus.cc:3037] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.903775 25402 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.904318 24127 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.904517 23862 raft_consensus.cc:2454] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbeb6a07fa7a44019ae00c169c414529 in term 1.
I20240501 14:10:27.905802 23584 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197). New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:27.913205 25393 raft_consensus.cc:482] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:27.913303 25393 raft_consensus.cc:504] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.913879 25393 leader_election.cc:288] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:27.914000 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:27.914080 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:27.914144 24259 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.914180 24127 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.914317 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:27.914438 23862 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.914604 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:27.914690 24328 leader_election.cc:302] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:27.914709 24523 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.914736 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:27.914829 23732 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.914934 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:27.915058 23995 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:27.915303 25393 raft_consensus.cc:2781] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:27.915401 25393 raft_consensus.cc:482] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:27.915452 25393 raft_consensus.cc:3037] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.916211 25393 raft_consensus.cc:504] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.916715 25393 leader_election.cc:288] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Requested vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:27.916828 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:27.916903 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:27.916924 24259 raft_consensus.cc:3037] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.916924 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:27.916999 23732 raft_consensus.cc:3037] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.916998 24127 raft_consensus.cc:3037] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.917747 24127 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.917754 24259 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.917754 23732 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.918177 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:27.918272 24523 raft_consensus.cc:3037] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.918335 24326 leader_election.cc:302] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 24d90b55560e45feb5fe4b9482317a4c, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:27.918367 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:27.918448 23862 raft_consensus.cc:3037] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.918555 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84da4677d89b42fe8c2418cd156c75ae" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:27.918637 23995 raft_consensus.cc:3037] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:27.919087 24523 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.919209 23862 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.919659 23995 raft_consensus.cc:2454] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:27.919811 25393 raft_consensus.cc:2781] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:27.919915 25393 raft_consensus.cc:686] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Becoming Leader. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Running, Role: LEADER
I20240501 14:10:27.920065 25393 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:27.922042 23584 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198). New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
W20240501 14:10:28.032465 25465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:28.032650 25465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:28.035573 25465 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240501 14:10:28.110741 23862 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.110757 24392 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.110824 24523 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.111070 23732 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.111104 24127 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.111451 25128 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.111598 25330 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.111924 25492 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.112193 24259 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.112708 25330 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.113176 25142 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.114501 23730 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.114526 24523 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.114873 25393 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.115924 25087 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.116554 24259 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.117503 24126 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.117955 25087 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.123131 23732 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.123703 25403 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.123775 24523 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.123883 24390 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.124094 25403 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.124331 24126 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.124689 25403 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.125138 23995 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.125396 23996 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.126143 25393 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.129709 25403 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.129813 23864 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.129813 23862 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.135560 25329 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.137136 25090 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.138443 25406 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.138847 25087 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.139189 25403 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.165149 25425 mvcc.cc:205] Tried to move back new op lower bound from 7022889484736364544 to 7022889483790446592. Current Snapshot: MvccSnapshot[applied={T|T < 7022889484736364544}]
INSERT report
rows total: 2000
time total: 191.961 ms
time per row: 0.0959806 ms
I20240501 14:10:28.273720 23584 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37374:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:28.274206 23584 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_5 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 14:10:28.278839 23711 tablet_service.cc:1467] Processing CreateTablet for tablet 697668f5caf34e63a9e5953c180959ee (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.279135 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.279124 24240 tablet_service.cc:1467] Processing CreateTablet for tablet 697668f5caf34e63a9e5953c180959ee (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.279408 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.279644 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 697668f5caf34e63a9e5953c180959ee (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.279896 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.280858 23842 tablet_service.cc:1467] Processing CreateTablet for tablet 21c8e4484fc94645869c7a912df55e4c (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.281085 23842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.281512 23974 tablet_service.cc:1467] Processing CreateTablet for tablet 697668f5caf34e63a9e5953c180959ee (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.281780 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.281880 24994 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:28.282697 24994 tablet_bootstrap.cc:654] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.283236 23976 tablet_service.cc:1467] Processing CreateTablet for tablet 21c8e4484fc94645869c7a912df55e4c (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.283495 23976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.283680 24503 tablet_service.cc:1467] Processing CreateTablet for tablet 697668f5caf34e63a9e5953c180959ee (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.283923 24503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.283933 23844 tablet_service.cc:1467] Processing CreateTablet for tablet a50dc90c18174aa289c75056c4c461b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.284030 24504 tablet_service.cc:1467] Processing CreateTablet for tablet 21c8e4484fc94645869c7a912df55e4c (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.284137 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.284241 24504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.285725 24108 tablet_service.cc:1467] Processing CreateTablet for tablet 697668f5caf34e63a9e5953c180959ee (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.285859 25018 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:28.285988 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.286047 23975 tablet_service.cc:1467] Processing CreateTablet for tablet a50dc90c18174aa289c75056c4c461b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.286242 23975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.286623 25018 tablet_bootstrap.cc:654] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.286652 24994 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:28.286742 24994 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 14:10:28.287170 24994 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.287256 24994 raft_consensus.cc:374] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.287276 24994 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.287384 24994 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.285876 24502 tablet_service.cc:1467] Processing CreateTablet for tablet a50dc90c18174aa289c75056c4c461b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.287629 24502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.287667 24994 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.287811 24994 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:28.287999 24998 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:28.288112 24106 tablet_service.cc:1467] Processing CreateTablet for tablet a50dc90c18174aa289c75056c4c461b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.288313 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.288565 24994 tablet_bootstrap.cc:654] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.288753 24998 tablet_bootstrap.cc:654] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.289376 24995 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:28.290102 25006 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:28.290176 24995 tablet_bootstrap.cc:654] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.290601 24996 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:28.290863 25006 tablet_bootstrap.cc:654] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.291026 24995 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:28.291100 24995 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.291541 24107 tablet_service.cc:1467] Processing CreateTablet for tablet 21c8e4484fc94645869c7a912df55e4c (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.291538 24995 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.291651 24995 raft_consensus.cc:374] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.291683 24995 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.291707 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.291787 24995 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.292011 24370 tablet_service.cc:1467] Processing CreateTablet for tablet 697668f5caf34e63a9e5953c180959ee (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.292066 24995 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.292104 24372 tablet_service.cc:1467] Processing CreateTablet for tablet a50dc90c18174aa289c75056c4c461b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.292237 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.292506 24371 tablet_service.cc:1467] Processing CreateTablet for tablet 21c8e4484fc94645869c7a912df55e4c (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.292701 25006 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:28.292713 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.292771 25006 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:10:28.293161 25006 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.293265 25006 raft_consensus.cc:374] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.293292 25006 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.293381 25006 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.293637 25006 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:28.293787 25006 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:28.294206 25449 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:28.294538 25006 tablet_bootstrap.cc:654] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.294906 25018 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:28.294966 25449 tablet_bootstrap.cc:654] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.294996 25018 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:10:28.295298 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.295464 25018 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.295575 25018 raft_consensus.cc:374] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.295607 25018 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.295722 25018 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.295981 25018 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.296128 25018 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:28.296818 25018 tablet_bootstrap.cc:654] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.297518 23711 tablet_service.cc:1467] Processing CreateTablet for tablet a50dc90c18174aa289c75056c4c461b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.297735 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.299444 23712 tablet_service.cc:1467] Processing CreateTablet for tablet 21c8e4484fc94645869c7a912df55e4c (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.299584 24995 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:28.299666 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.299937 24996 tablet_bootstrap.cc:654] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.300438 24995 tablet_bootstrap.cc:654] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.301021 24238 tablet_service.cc:1467] Processing CreateTablet for tablet 21c8e4484fc94645869c7a912df55e4c (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.301255 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.302237 24995 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:28.302313 24995 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:10:28.302313 24240 tablet_service.cc:1467] Processing CreateTablet for tablet a50dc90c18174aa289c75056c4c461b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.302528 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.302690 24995 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.302788 24995 raft_consensus.cc:374] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.302816 24995 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.302917 24995 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.303169 24995 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.303316 24995 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:28.304124 24995 tablet_bootstrap.cc:654] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.305544 24996 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:28.305626 24996 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240501 14:10:28.305694 24994 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:28.305778 24998 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:28.305784 24994 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 14:10:28.305843 24998 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240501 14:10:28.306036 24996 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.306146 24996 raft_consensus.cc:374] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.306178 24996 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.306157 24994 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.306170 24998 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.306254 24994 raft_consensus.cc:374] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.306281 24998 raft_consensus.cc:374] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.306283 24994 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.306308 24998 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.306304 24996 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.306385 24994 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.306414 24998 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.306586 24996 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.306643 24994 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.306658 24998 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.306757 24996 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:28.306777 24994 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:28.306798 24998 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:28.307510 24998 tablet_bootstrap.cc:654] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.307737 24994 tablet_bootstrap.cc:654] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.309800 25006 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:28.309888 25006 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240501 14:10:28.310142 25449 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:28.310222 25449 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.002s
I20240501 14:10:28.310571 24994 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:28.310550 25006 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.310653 24994 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:10:28.310664 25006 raft_consensus.cc:374] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.310621 25449 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.310734 25449 raft_consensus.cc:374] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.310766 25449 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.310868 25449 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.311038 24994 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.311123 25449 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:28.311153 24994 raft_consensus.cc:374] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.311189 24994 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.311275 25449 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:28.311298 24994 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.311630 24994 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.312124 25449 tablet_bootstrap.cc:654] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.310695 25006 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.312347 25006 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.312757 25006 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 14:10:28.312922 25006 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:28.316373 25006 tablet_bootstrap.cc:654] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.316468 24998 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:28.316555 24998 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 14:10:28.316948 24998 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.317060 24998 raft_consensus.cc:374] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.317085 24998 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.317184 24998 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.317425 24998 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.317580 24998 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:28.317639 24996 tablet_bootstrap.cc:654] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.318321 24998 tablet_bootstrap.cc:654] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.319092 24996 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:28.319170 24996 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240501 14:10:28.319550 24995 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:28.319635 24995 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s
I20240501 14:10:28.320010 24995 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.320119 24995 raft_consensus.cc:374] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.320144 24995 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.320272 24995 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.320466 24996 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.320552 24995 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:28.320565 24996 raft_consensus.cc:374] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.320596 24996 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.320724 25018 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:28.320801 25018 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240501 14:10:28.320744 24996 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.321038 24996 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.321185 24996 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:28.321278 25018 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.321403 25018 raft_consensus.cc:374] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.321434 25018 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.321555 25018 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.321851 25018 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.322520 25018 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:28.323376 25018 tablet_bootstrap.cc:654] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.324007 24996 tablet_bootstrap.cc:654] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.325189 25449 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:28.325264 25449 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240501 14:10:28.325642 25449 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.325726 24996 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:28.325757 25449 raft_consensus.cc:374] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.325783 25449 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.325788 24996 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:10:28.325883 25449 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.326139 25449 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:28.326155 24996 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.326252 24996 raft_consensus.cc:374] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.326282 24996 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.326395 24996 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.326691 25449 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:28.326915 24998 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:28.326996 24998 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 14:10:28.327260 24996 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.327414 24998 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.327520 24998 raft_consensus.cc:374] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.327554 24998 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.327685 24998 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.328035 24998 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.328656 25006 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:28.328735 25006 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240501 14:10:28.329140 25006 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.329252 25006 raft_consensus.cc:374] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.329284 25006 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.329386 25006 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.329660 25006 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:28.330394 25449 tablet_bootstrap.cc:654] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.330590 25018 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:28.330665 25018 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 14:10:28.331069 25018 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.331177 25018 raft_consensus.cc:374] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.331210 25018 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.331315 25018 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.331693 25018 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:28.332633 25449 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:28.332713 25449 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 14:10:28.333113 25449 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.333221 25449 raft_consensus.cc:374] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.333249 25449 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.333362 25449 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.333644 25449 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:28.350628 25036 raft_consensus.cc:482] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:28.350739 25036 raft_consensus.cc:504] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.352216 25036 leader_election.cc:288] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:28.357278 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:28.357410 24392 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:28.358568 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:28.358635 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:28.358695 24127 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:28.358747 24259 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:28.358893 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:28.359023 23994 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:28.359232 23798 leader_election.cc:302] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:28.359472 25036 raft_consensus.cc:2781] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:28.359527 25036 raft_consensus.cc:482] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:28.359565 25036 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.360002 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:28.360127 24523 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:28.360507 25036 raft_consensus.cc:504] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.361089 25036 leader_election.cc:288] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [CANDIDATE]: Term 1 election: Requested vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:28.361347 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:28.361413 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:28.361469 23994 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.361517 24259 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.361595 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:28.361685 24392 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.361657 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:28.361747 24127 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.362438 24259 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:28.362474 23994 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:28.362504 24392 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:28.362534 24127 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:28.362987 23797 leader_election.cc:302] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:28.363157 25036 raft_consensus.cc:2781] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:28.363180 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:28.363276 24523 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.363344 25036 raft_consensus.cc:686] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Becoming Leader. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Running, Role: LEADER
I20240501 14:10:28.363647 25036 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.364212 24523 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:28.364274 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:28.364414 23732 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:28.364274 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:28.364668 23731 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.365537 23731 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:28.366942 23585 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2502b9e8098b4bc5af7204835fce444b (127.6.153.194). New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:28.389286 25393 raft_consensus.cc:482] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:28.389372 25393 raft_consensus.cc:504] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.389911 25393 leader_election.cc:288] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:28.390041 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:28.390147 23864 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:28.390332 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:28.390359 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:28.390444 23994 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:28.390453 24127 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:28.390645 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:28.390769 23731 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 0.
I20240501 14:10:28.390767 24328 leader_election.cc:302] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:28.390923 25393 raft_consensus.cc:2781] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:28.390980 25393 raft_consensus.cc:482] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:28.391023 25393 raft_consensus.cc:3037] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.391741 25393 raft_consensus.cc:504] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.392269 25393 leader_election.cc:288] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Requested vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:28.392400 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:28.392501 24127 raft_consensus.cc:3037] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.392570 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:28.392616 24258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:28.392671 24524 raft_consensus.cc:3037] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.392725 24258 raft_consensus.cc:3037] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.392755 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:28.392838 23994 raft_consensus.cc:3037] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.393321 24127 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:28.393534 24258 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:28.393534 24524 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:28.393648 23994 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:28.393823 24327 leader_election.cc:302] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 40b69f74de5a447f86c1445f8e84850d, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:28.393918 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:28.393990 25390 raft_consensus.cc:2781] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:28.394026 24523 raft_consensus.cc:2362] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1. Re-sending same reply.
I20240501 14:10:28.394080 25390 raft_consensus.cc:686] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Becoming Leader. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Running, Role: LEADER
I20240501 14:10:28.394210 25390 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.394376 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:28.394476 24259 raft_consensus.cc:2362] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1. Re-sending same reply.
I20240501 14:10:28.394934 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:28.395027 23864 raft_consensus.cc:3037] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.395948 23864 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:28.396378 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50dc90c18174aa289c75056c4c461b0" candidate_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:28.396240 23585 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198). New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:28.396478 23731 raft_consensus.cc:3037] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.397260 23731 raft_consensus.cc:2454] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e6e9293a4614dabb5ad89fe3f4d5d04 in term 1.
I20240501 14:10:28.453565 25023 raft_consensus.cc:482] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:28.453667 25023 raft_consensus.cc:504] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.454334 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:28.454464 23864 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:28.454442 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:28.454557 23994 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:28.454589 25023 leader_election.cc:288] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:28.454882 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:28.454954 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:28.455020 24259 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:28.455001 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:28.455067 24127 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:28.455118 24392 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:28.455124 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:28.455235 24523 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 0.
I20240501 14:10:28.455287 23666 leader_election.cc:302] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:28.455462 25023 raft_consensus.cc:2781] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:28.455518 25023 raft_consensus.cc:482] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:28.455564 25023 raft_consensus.cc:3037] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.456315 25023 raft_consensus.cc:504] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.456775 25023 leader_election.cc:288] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 election: Requested vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:28.457029 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:28.457038 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:28.457124 24259 raft_consensus.cc:3037] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.457129 24127 raft_consensus.cc:3037] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.457121 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:28.457160 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:28.457219 24392 raft_consensus.cc:3037] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.457248 23864 raft_consensus.cc:3037] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.457973 24127 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:28.458009 24392 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:28.458014 24259 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:28.458025 23864 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:28.458298 23666 leader_election.cc:302] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:28.458374 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:28.458415 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697668f5caf34e63a9e5953c180959ee" candidate_uuid: "24d90b55560e45feb5fe4b9482317a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:28.458442 25023 raft_consensus.cc:2781] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:28.458501 24523 raft_consensus.cc:3037] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.458477 23994 raft_consensus.cc:3037] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.458549 25023 raft_consensus.cc:686] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Becoming Leader. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Running, Role: LEADER
I20240501 14:10:28.458683 25023 consensus_queue.cc:237] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.459399 23994 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:28.459406 24523 raft_consensus.cc:2454] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24d90b55560e45feb5fe4b9482317a4c in term 1.
I20240501 14:10:28.460891 23585 catalog_manager.cc:5484] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193). New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
W20240501 14:10:28.605554 25547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:28.605760 25547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:28.608791 25547 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240501 14:10:28.681128 24523 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.681810 23994 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.681810 23864 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.682227 25391 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.682523 23731 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.682507 25391 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.682746 25391 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.682930 24127 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.683001 25391 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.683341 25577 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.683422 24259 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.691995 25575 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.694509 24392 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.695051 23862 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.695659 24524 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.695760 24127 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.696588 24258 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.696943 25219 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.697371 23996 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.697350 25219 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.697619 25219 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.697871 25219 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.704550 25590 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.705144 25589 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.717803 23995 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.726964 25036 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.744338 24523 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.745609 25543 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.751034 24392 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.751385 25036 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.757637 23731 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.757994 25036 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.764374 24127 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.764732 25036 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:28.774530 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:28.774926 25543 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 175.24 ms
time per row: 0.0876201 ms
I20240501 14:10:28.829353 23588 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37374:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 14:10:28.830404 23588 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_6 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 14:10:28.839910 23712 tablet_service.cc:1467] Processing CreateTablet for tablet de25122a01e845d2b35e4a1023789d3e (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.840191 23712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.842525 23711 tablet_service.cc:1467] Processing CreateTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.842737 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.844710 23710 tablet_service.cc:1467] Processing CreateTablet for tablet 8d93a76aca9341088d460780323adbc3 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.844905 23710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d93a76aca9341088d460780323adbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.847643 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 8d93a76aca9341088d460780323adbc3 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.847885 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d93a76aca9341088d460780323adbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.847841 23844 tablet_service.cc:1467] Processing CreateTablet for tablet de25122a01e845d2b35e4a1023789d3e (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.848078 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.849151 23842 tablet_service.cc:1467] Processing CreateTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.849339 23842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.850100 25615 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:28.850750 25615 tablet_bootstrap.cc:654] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.850989 23975 tablet_service.cc:1467] Processing CreateTablet for tablet de25122a01e845d2b35e4a1023789d3e (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.851197 23975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.851565 25615 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:28.851624 25615 ts_tablet_manager.cc:1367] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.851630 23974 tablet_service.cc:1467] Processing CreateTablet for tablet 8d93a76aca9341088d460780323adbc3 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.851823 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d93a76aca9341088d460780323adbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.851886 25615 raft_consensus.cc:348] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.851979 25615 raft_consensus.cc:374] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.851998 25615 raft_consensus.cc:729] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.852072 25615 consensus_queue.cc:260] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.852284 25615 ts_tablet_manager.cc:1398] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:28.852391 25615 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:28.852957 25615 tablet_bootstrap.cc:654] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.853354 25617 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:28.853706 23976 tablet_service.cc:1467] Processing CreateTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.853794 25615 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:28.853863 25615 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.853906 23976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.854014 25617 tablet_bootstrap.cc:654] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.854239 25615 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.854336 25615 raft_consensus.cc:374] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.854362 25615 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.854444 25615 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.854707 25615 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.854779 25617 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:28.854833 25617 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.855109 25617 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.855185 25617 raft_consensus.cc:374] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.855203 25617 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.855295 25617 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.855643 25617 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.855793 25617 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:28.856014 24372 tablet_service.cc:1467] Processing CreateTablet for tablet de25122a01e845d2b35e4a1023789d3e (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.856050 24371 tablet_service.cc:1467] Processing CreateTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.856112 24370 tablet_service.cc:1467] Processing CreateTablet for tablet 8d93a76aca9341088d460780323adbc3 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.856290 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.856384 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.856609 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d93a76aca9341088d460780323adbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.857673 24240 tablet_service.cc:1467] Processing CreateTablet for tablet de25122a01e845d2b35e4a1023789d3e (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.857905 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.858431 24238 tablet_service.cc:1467] Processing CreateTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.858644 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.859406 25617 tablet_bootstrap.cc:654] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.859970 25622 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:28.860260 24239 tablet_service.cc:1467] Processing CreateTablet for tablet 8d93a76aca9341088d460780323adbc3 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.860457 24239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d93a76aca9341088d460780323adbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.860723 25622 tablet_bootstrap.cc:654] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.860793 24107 tablet_service.cc:1467] Processing CreateTablet for tablet de25122a01e845d2b35e4a1023789d3e (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.861037 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.862012 25622 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:28.862087 25622 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.862560 24106 tablet_service.cc:1467] Processing CreateTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.862633 25622 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.862731 25622 raft_consensus.cc:374] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.862761 25622 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.862783 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.862869 25622 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.863189 25622 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.863324 25622 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:28.864131 25622 tablet_bootstrap.cc:654] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.864775 24108 tablet_service.cc:1467] Processing CreateTablet for tablet 8d93a76aca9341088d460780323adbc3 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.864989 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d93a76aca9341088d460780323adbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.865257 25622 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:28.865340 25622 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:28.865744 25622 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.865864 25622 raft_consensus.cc:374] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.865903 25622 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.866030 25622 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.866132 25617 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:28.866219 25617 ts_tablet_manager.cc:1367] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240501 14:10:28.866283 25622 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.866448 25622 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:28.867177 25622 tablet_bootstrap.cc:654] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.866618 25617 raft_consensus.cc:348] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.867621 25617 raft_consensus.cc:374] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.867592 24502 tablet_service.cc:1467] Processing CreateTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.867674 25617 raft_consensus.cc:729] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.867835 24502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.867792 25617 consensus_queue.cc:260] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.867991 25613 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:28.868345 25617 ts_tablet_manager.cc:1398] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:28.868845 25613 tablet_bootstrap.cc:654] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.868579 25617 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:28.870029 24503 tablet_service.cc:1467] Processing CreateTablet for tablet 8d93a76aca9341088d460780323adbc3 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.870231 24503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d93a76aca9341088d460780323adbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.870703 25617 tablet_bootstrap.cc:654] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.871387 25628 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:28.872117 25628 tablet_bootstrap.cc:654] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.867244 24504 tablet_service.cc:1467] Processing CreateTablet for tablet de25122a01e845d2b35e4a1023789d3e (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 14:10:28.872982 24504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:28.873231 25628 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:28.873315 25628 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.873708 25628 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.873836 25628 raft_consensus.cc:374] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.873870 25628 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.873989 25628 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.874243 25628 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:28.874398 25628 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:28.875422 25628 tablet_bootstrap.cc:654] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.876192 25615 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:28.876344 25628 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:28.876422 25628 ts_tablet_manager.cc:1367] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 14:10:28.876765 25628 raft_consensus.cc:348] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.876856 25628 raft_consensus.cc:374] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.876873 25615 tablet_bootstrap.cc:654] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.876893 25628 raft_consensus.cc:729] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.877012 25628 consensus_queue.cc:260] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.877269 25628 ts_tablet_manager.cc:1398] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:28.877553 25628 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: Bootstrap starting.
I20240501 14:10:28.878144 25620 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:28.878216 25628 tablet_bootstrap.cc:654] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.878868 25620 tablet_bootstrap.cc:654] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.879153 25628 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: No bootstrap required, opened a new log
I20240501 14:10:28.879228 25628 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 14:10:28.879688 25624 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:28.879650 25628 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.879765 25628 raft_consensus.cc:374] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.879796 25628 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.879909 25628 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.880286 25628 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.880474 25624 tablet_bootstrap.cc:654] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.881181 25622 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529: No bootstrap required, opened a new log
I20240501 14:10:28.881217 25615 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b: No bootstrap required, opened a new log
I20240501 14:10:28.881266 25622 ts_tablet_manager.cc:1367] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240501 14:10:28.881300 25624 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:28.881299 25615 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:10:28.881371 25624 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.881676 25622 raft_consensus.cc:348] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.881784 25622 raft_consensus.cc:374] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.881732 25624 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.881814 25622 raft_consensus.cc:729] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.881737 25615 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.881834 25624 raft_consensus.cc:374] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.881856 25615 raft_consensus.cc:374] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.881863 25624 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.881880 25615 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.881938 25622 consensus_queue.cc:260] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.882005 25615 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.881996 25624 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.882202 25622 ts_tablet_manager.cc:1398] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.882256 25624 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.882265 25615 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.882390 25624 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:28.882716 25620 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:28.882812 25620 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 14:10:28.883131 25624 tablet_bootstrap.cc:654] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.883229 25620 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.883355 25620 raft_consensus.cc:374] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.883385 25620 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.883522 25620 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.883801 25620 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:28.884393 25620 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:28.885300 25620 tablet_bootstrap.cc:654] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.885903 25624 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:28.886029 25624 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 14:10:28.886585 25624 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.886749 25624 raft_consensus.cc:374] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.886768 25617 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd: No bootstrap required, opened a new log
I20240501 14:10:28.886829 25624 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.886868 25617 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.001s
I20240501 14:10:28.886965 25613 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:28.887069 25613 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20240501 14:10:28.887053 25624 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.887277 25617 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.887388 25620 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:28.887424 25617 raft_consensus.cc:374] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.887465 25617 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.887511 25620 ts_tablet_manager.cc:1367] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 14:10:28.887524 25613 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.887578 25617 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.887660 25613 raft_consensus.cc:374] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.887697 25613 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.887848 25617 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.887938 25624 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.887916 25620 raft_consensus.cc:348] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.887948 25613 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.888041 25620 raft_consensus.cc:374] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.888073 25620 raft_consensus.cc:729] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.888242 25624 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:28.888765 25613 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:28.888976 25613 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:28.889195 25624 tablet_bootstrap.cc:654] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.889982 25613 tablet_bootstrap.cc:654] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.890235 25624 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: No bootstrap required, opened a new log
I20240501 14:10:28.890357 25624 ts_tablet_manager.cc:1367] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:28.890843 25624 raft_consensus.cc:348] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.891064 25624 raft_consensus.cc:374] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.891369 25624 raft_consensus.cc:729] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.891595 25620 consensus_queue.cc:260] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.891707 25613 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:28.891616 25624 consensus_queue.cc:260] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.891778 25613 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 14:10:28.891939 25620 ts_tablet_manager.cc:1398] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 14:10:28.892083 25620 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04: Bootstrap starting.
I20240501 14:10:28.892159 25613 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.892264 25613 raft_consensus.cc:374] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.892298 25613 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.892412 25613 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.892544 25624 ts_tablet_manager.cc:1398] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:28.892671 25613 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:28.892828 25613 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c: Bootstrap starting.
I20240501 14:10:28.893594 25613 tablet_bootstrap.cc:654] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.893898 25620 tablet_bootstrap.cc:654] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04: Neither blocks nor log segments found. Creating new log.
I20240501 14:10:28.894798 25613 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c: No bootstrap required, opened a new log
I20240501 14:10:28.894881 25613 ts_tablet_manager.cc:1367] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:28.895269 25613 raft_consensus.cc:348] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.895478 25613 raft_consensus.cc:374] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.895520 25613 raft_consensus.cc:729] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d90b55560e45feb5fe4b9482317a4c, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.895653 25613 consensus_queue.cc:260] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.896018 25613 ts_tablet_manager.cc:1398] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:28.896135 25620 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04: No bootstrap required, opened a new log
I20240501 14:10:28.896203 25620 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 14:10:28.896596 25620 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.896694 25620 raft_consensus.cc:374] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 14:10:28.896724 25620 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Initialized, Role: FOLLOWER
I20240501 14:10:28.896857 25620 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.897179 25620 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 14:10:28.937531 25326 raft_consensus.cc:482] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:28.937641 25326 raft_consensus.cc:504] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.938297 25326 leader_election.cc:288] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:28.938377 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:28.938526 23732 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 0.
I20240501 14:10:28.938547 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:28.938653 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:28.938694 23862 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 0.
I20240501 14:10:28.938737 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:28.938769 24259 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 0.
I20240501 14:10:28.938858 23995 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 0.
I20240501 14:10:28.939060 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:28.939179 24392 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 0.
I20240501 14:10:28.939172 24458 leader_election.cc:302] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 40b69f74de5a447f86c1445f8e84850d, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:28.939291 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:28.939363 25326 raft_consensus.cc:2781] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:28.939409 25326 raft_consensus.cc:482] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:28.939445 25326 raft_consensus.cc:3037] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.939438 24128 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 0.
I20240501 14:10:28.940359 25326 raft_consensus.cc:504] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.940834 25326 leader_election.cc:288] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [CANDIDATE]: Term 1 election: Requested vote from peers 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:28.940979 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:28.941041 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:28.941084 23732 raft_consensus.cc:3037] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.941140 24259 raft_consensus.cc:3037] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.941257 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:28.941344 24392 raft_consensus.cc:3037] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.941372 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:28.941489 23862 raft_consensus.cc:3037] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.942016 23732 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 1.
I20240501 14:10:28.942072 24259 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 1.
I20240501 14:10:28.942200 24392 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 1.
I20240501 14:10:28.942380 23862 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 1.
I20240501 14:10:28.942443 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:28.942453 24457 leader_election.cc:302] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 40b69f74de5a447f86c1445f8e84850d, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:28.942523 23995 raft_consensus.cc:3037] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.942699 25326 raft_consensus.cc:2781] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:28.942893 25326 raft_consensus.cc:686] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 LEADER]: Becoming Leader. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Running, Role: LEADER
I20240501 14:10:28.943204 25326 consensus_queue.cc:237] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:28.943429 23995 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 1.
I20240501 14:10:28.943697 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "182d85ec9c03441fb974ee50ad36ea8b" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:28.943794 24128 raft_consensus.cc:3037] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.944694 24128 raft_consensus.cc:2454] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 1.
I20240501 14:10:28.946250 23588 catalog_manager.cc:5484] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d reported cstate change: term changed from 0 to 1, leader changed from <none> to 40b69f74de5a447f86c1445f8e84850d (127.6.153.199). New cstate: current_term: 1 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:28.955564 25512 raft_consensus.cc:482] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:28.955641 25512 raft_consensus.cc:504] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.956225 25512 leader_election.cc:288] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:28.956394 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:28.956418 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:28.956503 24523 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:28.956523 24392 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:28.956642 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b" is_pre_election: true
I20240501 14:10:28.956743 23862 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:28.956871 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:28.956971 23995 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:28.956974 24064 leader_election.cc:302] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 40b69f74de5a447f86c1445f8e84850d, 7e6e9293a4614dabb5ad89fe3f4d5d04; no voters:
I20240501 14:10:28.957137 25512 raft_consensus.cc:2781] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:28.957191 25512 raft_consensus.cc:482] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:28.957163 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:28.957238 25512 raft_consensus.cc:3037] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.957266 24259 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:28.957244 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:28.957350 23732 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 0.
I20240501 14:10:28.957998 25512 raft_consensus.cc:504] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.958580 25512 leader_election.cc:288] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [CANDIDATE]: Term 1 election: Requested vote from peers dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:28.958658 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:28.958700 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:28.958717 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:28.958747 24392 raft_consensus.cc:3037] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.958778 24259 raft_consensus.cc:3037] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.958803 24523 raft_consensus.cc:3037] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.959317 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:28.959436 23862 raft_consensus.cc:3037] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.959587 24392 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:28.959587 24259 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:28.959692 24523 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:28.959964 24063 leader_election.cc:302] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 40b69f74de5a447f86c1445f8e84850d, 7e6e9293a4614dabb5ad89fe3f4d5d04, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:28.960100 25512 raft_consensus.cc:2781] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:28.960178 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:28.960270 23732 raft_consensus.cc:3037] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.960268 25512 raft_consensus.cc:686] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Becoming Leader. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Running, Role: LEADER
I20240501 14:10:28.960289 23862 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:28.960403 25512 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:28.960489 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de25122a01e845d2b35e4a1023789d3e" candidate_uuid: "166d7170d98944e88de2eccda9d30545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:28.960579 23995 raft_consensus.cc:3037] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:28.961155 23732 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:28.961328 23995 raft_consensus.cc:2454] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d7170d98944e88de2eccda9d30545 in term 1.
I20240501 14:10:28.962591 23581 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 reported cstate change: term changed from 0 to 1, leader changed from <none> to 166d7170d98944e88de2eccda9d30545 (127.6.153.196). New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:29.012574 25612 raft_consensus.cc:482] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 14:10:29.012686 25612 raft_consensus.cc:504] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:29.013242 25612 leader_election.cc:288] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:29.013456 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c" is_pre_election: true
I20240501 14:10:29.013491 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" is_pre_election: true
I20240501 14:10:29.013584 23732 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:29.013629 24392 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:29.013633 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" is_pre_election: true
I20240501 14:10:29.013644 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d" is_pre_election: true
I20240501 14:10:29.013738 23995 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:29.013763 24523 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:29.014045 23798 leader_election.cc:302] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:29.014200 25612 raft_consensus.cc:2781] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 14:10:29.014254 25612 raft_consensus.cc:482] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 14:10:29.014288 25612 raft_consensus.cc:3037] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:29.014240 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545" is_pre_election: true
I20240501 14:10:29.014304 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529" is_pre_election: true
I20240501 14:10:29.014400 24128 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:29.014421 24259 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 0.
I20240501 14:10:29.015259 25612 raft_consensus.cc:504] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:29.015843 25612 leader_election.cc:288] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [CANDIDATE]: Term 1 election: Requested vote from peers 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:29.016018 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:29.016018 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:29.016108 24392 raft_consensus.cc:3037] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:29.016119 24128 raft_consensus.cc:3037] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:29.016098 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:29.016180 23995 raft_consensus.cc:3037] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:29.016486 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:29.016583 23732 raft_consensus.cc:3037] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:29.016993 24392 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:29.016999 24128 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:29.016999 23995 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:29.017393 23797 leader_election.cc:302] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd; no voters:
I20240501 14:10:29.017400 24523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:29.017462 23732 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:29.017521 24523 raft_consensus.cc:3037] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:29.017539 25612 raft_consensus.cc:2781] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Leader election won for term 1
I20240501 14:10:29.017644 25612 raft_consensus.cc:686] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Becoming Leader. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Running, Role: LEADER
I20240501 14:10:29.017777 25612 consensus_queue.cc:237] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:29.017944 24259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d93a76aca9341088d460780323adbc3" candidate_uuid: "2502b9e8098b4bc5af7204835fce444b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:29.018052 24259 raft_consensus.cc:3037] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 0 FOLLOWER]: Advancing to term 1
I20240501 14:10:29.018395 24523 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:29.018847 24259 raft_consensus.cc:2454] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2502b9e8098b4bc5af7204835fce444b in term 1.
I20240501 14:10:29.019984 23581 catalog_manager.cc:5484] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2502b9e8098b4bc5af7204835fce444b (127.6.153.194). New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
W20240501 14:10:29.161590 25649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:29.161851 25649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:29.164937 25649 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240501 14:10:29.250304 23995 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.255019 25646 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.255362 23732 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.256177 24392 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.256945 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.257398 25036 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.258466 24128 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.259488 25644 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.259600 24390 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.260046 23862 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.260146 24259 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.260520 23996 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.260684 23993 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.260802 23731 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.261425 25612 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.262689 24127 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.263607 24523 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.264734 25686 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.265146 25686 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.265604 25686 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.265799 25686 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.269049 24259 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.269915 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.273699 25644 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.281832 25036 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.282141 25036 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.294986 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.295202 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.295451 24258 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.295481 25646 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.295598 25512 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.295815 25512 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.296314 24523 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.296655 25512 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 14:10:29.297907 23864 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 14:10:29.303418 25646 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 205.328 ms
time per row: 0.102664 ms
I20240501 14:10:29.411573 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.201:36409
--local_ip_for_outbound_sockets=127.6.153.201
--tserver_master_addrs=127.6.153.254:34031
--webserver_port=36785
--webserver_interface=127.6.153.201
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 14:10:29.514221 25728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:29.514422 25728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:29.514498 25728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:29.517496 25728 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:29.517555 25728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:29.517634 25728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.201
I20240501 14:10:29.521171 25728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.201:36409
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.6.153.201
--webserver_port=36785
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.25728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.201
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:29.522130 25728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:29.523149 25728 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:29.531042 25736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:29.531365 25733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:29.531770 25734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:29.532253 25728 server_base.cc:1034] running on GCE node
I20240501 14:10:29.532588 25728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:29.533077 25728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:29.534224 25728 hybrid_clock.cc:648] HybridClock initialized: now 1714572629534208 us; error 30 us; skew 500 ppm
I20240501 14:10:29.536082 25728 webserver.cc:460] Webserver started at http://127.6.153.201:36785/ using document root <none> and password file <none>
I20240501 14:10:29.536652 25728 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:29.536728 25728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:29.539912 25728 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:10:29.541709 25742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.542809 25728 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:29.542909 25728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "a308c0ed71f44f57967c97239d1198f7"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:29.543241 25728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:29.553994 25728 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:29.554530 25728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:29.554659 25728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:29.555145 25728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:29.556071 25728 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:29.556133 25728 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.556180 25728 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:29.556205 25728 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.565107 25728 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.201:36409
I20240501 14:10:29.565253 25855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.201:36409 every 8 connection(s)
I20240501 14:10:29.566236 25728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 14:10:29.566812 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 25728
I20240501 14:10:29.567250 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.202:34929
--local_ip_for_outbound_sockets=127.6.153.202
--tserver_master_addrs=127.6.153.254:34031
--webserver_port=39461
--webserver_interface=127.6.153.202
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:29.581216 25856 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:29.581422 25856 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:29.581913 25856 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:29.582899 23589 ts_manager.cc:194] Re-registered known tserver with Master: a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409)
I20240501 14:10:29.583472 23589 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.201:46453
W20240501 14:10:29.684428 25858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:29.684650 25858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:29.684707 25858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:29.687927 25858 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:29.687983 25858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:29.688057 25858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.202
I20240501 14:10:29.691571 25858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.202:34929
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.6.153.202
--webserver_port=39461
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.25858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.202
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:29.692351 25858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:29.693397 25858 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:29.700940 25867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:29.701197 25865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:29.701570 25864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:29.701962 25858 server_base.cc:1034] running on GCE node
I20240501 14:10:29.702328 25858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:29.702877 25858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:29.704048 25858 hybrid_clock.cc:648] HybridClock initialized: now 1714572629704031 us; error 27 us; skew 500 ppm
I20240501 14:10:29.705715 25858 webserver.cc:460] Webserver started at http://127.6.153.202:39461/ using document root <none> and password file <none>
I20240501 14:10:29.706219 25858 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:29.706286 25858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:29.709600 25858 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:29.711387 25873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.712576 25858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:29.712728 25858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "c958143102d04279ac25aa56214d0d4b"
format_stamp: "Formatted at 2024-05-01 14:10:22 on dist-test-slave-cmf0"
I20240501 14:10:29.713092 25858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:29.725579 25858 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:29.726112 25858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:29.726240 25858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:29.726693 25858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:29.727512 25858 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:29.727557 25858 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.727604 25858 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:29.727630 25858 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.736605 25858 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.202:34929
I20240501 14:10:29.736644 25986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.202:34929 every 8 connection(s)
I20240501 14:10:29.737557 25858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240501 14:10:29.743163 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 25858
I20240501 14:10:29.743633 6759 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxMccU0/build/debug/bin/kudu
/tmp/dist-test-taskxMccU0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.153.200:37319
--local_ip_for_outbound_sockets=127.6.153.200
--tserver_master_addrs=127.6.153.254:34031
--webserver_port=38521
--webserver_interface=127.6.153.200
--builtin_ntp_servers=127.6.153.212:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 14:10:29.747923 25987 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:29.748121 25987 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:29.748658 25987 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:29.749621 23589 ts_manager.cc:194] Re-registered known tserver with Master: c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929)
I20240501 14:10:29.750294 23589 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.202:43153
W20240501 14:10:29.851163 25990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 14:10:29.851351 25990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 14:10:29.851399 25990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 14:10:29.854285 25990 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 14:10:29.854341 25990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 14:10:29.854408 25990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.153.200
I20240501 14:10:29.858165 25990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.153.212:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.153.200:37319
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.6.153.200
--webserver_port=38521
--tserver_master_addrs=127.6.153.254:34031
--never_fsync=true
--heap_profile_path=/tmp/kudu.25990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.153.200
--log_dir=/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 13:43:12 UTC on 1a15035a6830
build id 73
I20240501 14:10:29.858853 25990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 14:10:29.859859 25990 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 14:10:29.867291 25995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:29.867619 25996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 14:10:29.867959 25998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 14:10:29.868371 25990 server_base.cc:1034] running on GCE node
I20240501 14:10:29.868670 25990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 14:10:29.869138 25990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 14:10:29.870275 25990 hybrid_clock.cc:648] HybridClock initialized: now 1714572629870267 us; error 43 us; skew 500 ppm
I20240501 14:10:29.871929 25990 webserver.cc:460] Webserver started at http://127.6.153.200:38521/ using document root <none> and password file <none>
I20240501 14:10:29.872423 25990 fs_manager.cc:359] Metadata directory not provided
I20240501 14:10:29.872488 25990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 14:10:29.875586 25990 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 14:10:29.877460 26004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.878468 25990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:29.878557 25990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "6d832f3f723641778e60b3fd855ff98e"
format_stamp: "Formatted at 2024-05-01 14:10:21 on dist-test-slave-cmf0"
I20240501 14:10:29.878981 25990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 14:10:29.891120 25990 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 14:10:29.891734 25990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 14:10:29.891870 25990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 14:10:29.892390 25990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 14:10:29.893308 25990 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 14:10:29.893369 25990 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.893419 25990 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 14:10:29.893445 25990 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 14:10:29.902715 25990 rpc_server.cc:307] RPC server started. Bound to: 127.6.153.200:37319
I20240501 14:10:29.902784 26117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.153.200:37319 every 8 connection(s)
I20240501 14:10:29.903883 25990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 14:10:29.911542 6759 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxMccU0/build/debug/bin/kudu as pid 25990
I20240501 14:10:29.912825 26118 heartbeater.cc:347] Connected to a master server at 127.6.153.254:34031
I20240501 14:10:29.913026 26118 heartbeater.cc:464] Registering TS with master...
I20240501 14:10:29.913563 26118 heartbeater.cc:511] Master 127.6.153.254:34031 requested a full tablet report, sending...
I20240501 14:10:29.914573 23589 ts_manager.cc:194] Re-registered known tserver with Master: 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319)
I20240501 14:10:29.915127 23589 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.153.200:57197
I20240501 14:10:30.106300 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:30.106359 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.106354 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:30.117249 24504 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:30.133510 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:30.136175 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:30.139158 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.144076 23844 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:30.144344 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:30.154863 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.264434 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:30.265817 23844 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:30.265818 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:30.274716 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:30.277117 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:30.284060 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.288358 24504 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:30.311883 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.319653 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:30.319658 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.442677 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.444168 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:30.447046 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:30.451912 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:30.452505 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.458063 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:30.458163 24504 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:30.460356 23844 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:30.460461 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:30.467710 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:30.527002 24392 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:30.528718 24524 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.528781 24258 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.528781 24126 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.529081 23994 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.529130 23864 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.529394 25577 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.529599 26271 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.529742 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.530083 26271 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.530328 25577 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.530450 26271 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.532125 26271 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.533857 25577 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.534330 23994 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.536841 24258 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.537128 23864 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.537128 23731 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.538060 23590 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:30.540792 24524 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.542152 24126 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.548090 24392 consensus_queue.cc:237] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } }
I20240501 14:10:30.549574 24524 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.549607 24126 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.549607 24258 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.549940 26270 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.550091 23864 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.550112 26271 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.550235 25577 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.550470 26270 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.550732 23731 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.551031 23994 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.551379 26271 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.551710 26272 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.553718 26272 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } }
I20240501 14:10:30.555447 23731 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } }
I20240501 14:10:30.555863 23864 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } }
I20240501 14:10:30.556108 24126 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } }
I20240501 14:10:30.559161 23589 catalog_manager.cc:5484] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } }
I20240501 14:10:30.559978 23994 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } }
I20240501 14:10:30.560250 24524 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } }
I20240501 14:10:30.561390 24258 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } }
I20240501 14:10:30.567428 24392 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:30.573176 24258 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.567428 24390 consensus_queue.cc:237] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:30.573590 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 14:10:30.575402 24325 consensus_peers.cc:485] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 777ba3b6be404d12bb0b219c66da71a6. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.575661 23994 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.575834 23864 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.576124 23862 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.576256 24523 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.576280 24126 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.575661 23995 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.576556 24127 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.576658 24258 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.576712 25577 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.576822 26270 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.576938 26272 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.577189 24524 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.577553 23731 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 14:10:30.577683 24325 consensus_peers.cc:485] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fccb18388fb244bcb1dfd85908c0c118. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.577865 26272 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.578145 26272 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.578557 26270 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.578860 26270 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.578997 26302 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.579360 26302 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.579520 26271 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.581539 26272 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.581822 26272 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.582955 26271 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.584018 26272 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.584239 24127 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:30.585150 23574 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet fccb18388fb244bcb1dfd85908c0c118 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.585217 25856 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
I20240501 14:10:30.585475 23994 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.585978 23862 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.586860 24260 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.587390 24392 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.587220 23731 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.587981 23730 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.587985 24127 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.588040 24522 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.588496 24126 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.589567 26122 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.590065 26309 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.590354 26122 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.590888 23590 catalog_manager.cc:5484] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.591863 23581 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.592172 23994 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.592567 26263 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.593631 23862 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.593943 26263 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.594002 24524 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.593906 24523 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.594298 26263 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.595854 23574 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 777ba3b6be404d12bb0b219c66da71a6 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.596187 23864 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.597873 23730 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.597959 24260 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.598347 26263 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.600314 24260 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.600404 23994 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.602737 23584 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:30.604395 23730 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.605342 23862 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.605362 24524 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.609468 24258 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.611752 24126 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:30.612953 23995 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.614125 24392 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.614567 26263 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.615432 24524 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.615451 23731 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.615823 26308 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.616469 23994 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.616789 26122 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.617066 26122 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.616966 24392 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.617460 23862 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.617806 26122 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.618592 24258 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.619072 26263 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.619426 26310 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.619727 23862 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.621003 23731 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.621785 24524 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.621958 23994 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.623380 23584 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.624392 24258 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.625089 23576 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet fee07797b5c84ad8aa71d10c1ac24fe1 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.627183 24390 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
W20240501 14:10:30.632122 24061 consensus_peers.cc:485] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fee07797b5c84ad8aa71d10c1ac24fe1. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.633379 24258 consensus_queue.cc:237] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:30.635047 23994 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.635445 23731 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.635669 24390 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.636158 26325 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.636647 26325 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.636865 26326 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.637094 23862 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.638283 25626 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.639122 24126 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.639497 24524 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.639494 26326 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.639941 26325 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.640190 25626 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.640857 23864 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.640975 24524 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.641301 24126 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.642364 23994 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.642551 24390 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.643311 23582 catalog_manager.cc:5484] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.644704 23731 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.649235 24258 consensus_queue.cc:237] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:30.650995 24524 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.651114 23994 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.651125 23864 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.651496 24126 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.651621 26339 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.651738 26326 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.651846 26338 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.651989 26324 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.652485 24390 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.653082 26174 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.653266 23731 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.653803 26174 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.654729 26174 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.656123 23864 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.656574 23575 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 158b5efe75ce450abd20149d89157edc with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.657491 23584 catalog_manager.cc:5484] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:30.657851 24523 consensus_queue.cc:237] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:30.658179 23994 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.659206 24390 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.660913 24126 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.662276 24127 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.662861 23994 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.662894 24524 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.663245 23864 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.663741 23731 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.663987 23730 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.664495 25687 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.664609 24390 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.664839 25687 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.665122 25687 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.665431 25687 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.665702 25687 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.666040 24258 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:10:30.668380 24193 consensus_peers.cc:485] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 158b5efe75ce450abd20149d89157edc. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.668799 26345 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.669481 26345 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.671157 23994 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.671236 23864 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.671720 23586 catalog_manager.cc:5484] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:30.672713 24390 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.673214 24258 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.676753 24524 consensus_queue.cc:237] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:30.678741 23731 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.679181 23994 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.679440 24127 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.679565 24258 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.679589 26343 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.679859 24390 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.679939 26343 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.679992 23864 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.680215 25687 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.680457 25687 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.680430 23731 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.680691 25687 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.681265 24127 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.680869 26346 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20240501 14:10:30.683238 26353 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.683862 24127 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.685051 23994 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.685063 23731 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.685232 23576 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 182d85ec9c03441fb974ee50ad36ea8b with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.685956 23582 catalog_manager.cc:5484] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:30.686448 24258 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.688537 24390 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.691602 23864 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
W20240501 14:10:30.693742 24457 consensus_peers.cc:485] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 182d85ec9c03441fb974ee50ad36ea8b. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.698256 23994 consensus_queue.cc:237] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:30.699775 24258 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.699880 24524 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.700295 26361 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.700364 24390 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.700446 26304 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.700495 23731 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.700794 23864 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.700852 26304 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.701086 24127 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.701144 25492 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.701412 26304 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.703028 26300 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.703511 26365 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.703859 24390 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.704159 24127 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.706138 23864 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.706444 23588 catalog_manager.cc:5484] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.707136 24524 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.710786 24258 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.711251 23994 consensus_queue.cc:237] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:30.711433 23731 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.713290 24390 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.713714 24524 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.713856 24127 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.714071 23864 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.714190 26361 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.714398 26300 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.714546 26361 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.715422 24260 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.715613 26361 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.716686 26300 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.717321 23731 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.717499 26361 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.717746 26304 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.718308 24524 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.719377 23577 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 207a860a62e34c619ecec37f1eacaacd with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.719810 24127 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.719944 24390 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.720331 23864 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.721901 24260 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.722266 23586 catalog_manager.cc:5484] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.723881 23731 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.731772 23864 consensus_queue.cc:237] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:30.733325 24390 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.733410 24127 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.733513 23731 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.733790 24260 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.733778 26378 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.734138 24524 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.734359 26378 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.734719 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.734763 23994 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.735004 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.735313 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20240501 14:10:30.737510 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.739424 26378 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
W20240501 14:10:30.741645 23929 consensus_peers.cc:485] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 207a860a62e34c619ecec37f1eacaacd. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.741524 24390 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.742065 24523 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.742269 24127 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.742336 23588 catalog_manager.cc:5484] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:30.743676 23731 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.746201 23864 consensus_queue.cc:237] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:30.747073 23994 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.749815 24126 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.749905 24524 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.750331 24390 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.750591 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.750694 23994 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.750833 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.751529 23731 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.751821 26378 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.752070 26378 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.752480 25543 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.752543 25987 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
I20240501 14:10:30.754349 24258 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.754813 26380 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.755474 24258 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:30.756417 25543 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.757270 24127 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.757596 23731 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.759832 23581 catalog_manager.cc:5484] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.760524 23577 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 8d93a76aca9341088d460780323adbc3 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.760668 24392 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.760926 23994 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.762418 24258 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:30.764326 24523 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
W20240501 14:10:30.770855 23797 consensus_peers.cc:485] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d93a76aca9341088d460780323adbc3. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.772190 23731 consensus_queue.cc:237] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:30.773527 24523 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.773849 24127 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.773910 24392 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.773967 24258 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.774201 23994 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.774272 25589 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.774324 23864 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.774582 25589 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.774749 26399 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.774902 26397 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.775100 25589 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.775425 25589 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.777103 26400 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.777531 23994 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.779536 24258 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.779693 24127 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.781656 23864 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.781886 23581 catalog_manager.cc:5484] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.784089 24523 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.785888 24392 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.787786 23731 consensus_queue.cc:237] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:30.789561 24258 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.790006 26397 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.790181 24127 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.790226 23864 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.790258 23994 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.790627 26399 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.790781 26400 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.790910 26398 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.791189 24524 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.791711 24392 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.792299 26397 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.792596 26397 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.793437 26399 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.793853 24127 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.794489 24524 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.795450 23575 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 697668f5caf34e63a9e5953c180959ee with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.795433 24258 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.796383 23994 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.796622 23864 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.796926 23581 catalog_manager.cc:5484] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.800333 24392 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.803679 23994 consensus_queue.cc:237] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:30.805184 24258 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.805554 24524 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.805671 26374 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.805711 24392 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 14:10:30.805711 23665 consensus_peers.cc:485] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 697668f5caf34e63a9e5953c180959ee. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.806039 26374 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.806181 24127 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.806277 26304 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.806577 25492 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.806931 23864 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.807210 26374 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.807477 23731 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.807823 25492 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.809535 26374 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.809816 24127 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.809998 23731 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.810690 24524 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.811570 24392 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.811658 23864 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.813266 23582 catalog_manager.cc:5484] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:30.814345 24258 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.816584 23994 consensus_queue.cc:237] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:30.818615 23731 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.818809 23864 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 14:10:30.819195 23929 consensus_peers.cc:485] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be60ec7c4b0a46c28e0b23b6facae47e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.819252 26365 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.819444 24258 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.819429 26373 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.819729 24524 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.820029 24127 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.820096 26300 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.820345 26365 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.820619 24392 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.820601 26300 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.820958 26300 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.823236 26365 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.824097 23731 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.824947 23577 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet be60ec7c4b0a46c28e0b23b6facae47e with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.825675 23584 catalog_manager.cc:5484] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:30.825314 23864 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.825953 24258 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.826844 24127 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.827800 24392 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.827800 24524 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:30.833793 24392 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:30.835157 24258 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.835232 24127 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.835510 24524 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.835619 26272 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.835793 23731 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.835935 26272 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.836064 23994 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.836223 26307 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.836372 23864 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:30.836635 26307 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.836916 26307 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.837167 26307 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:30.838905 26302 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.839268 23864 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.840047 23994 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.840047 24524 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.841393 24258 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.841847 23588 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.844349 24127 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.846829 23730 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:30.847504 24392 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:30.850823 23994 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.851069 24127 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.851213 26301 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.851470 24258 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.851686 26301 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.851805 26302 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.851912 23730 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.852097 24524 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:30.852468 26301 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.853103 26301 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.853665 23864 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 14:10:30.853994 24325 consensus_peers.cc:485] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a50dc90c18174aa289c75056c4c461b0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:30.854094 26270 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:30.854195 26307 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.854712 23994 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.854941 23864 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.855566 23730 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.856408 23574 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a50dc90c18174aa289c75056c4c461b0 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:30.856624 24258 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.857540 23590 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.858434 24127 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.860853 24524 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:30.917569 26118 heartbeater.cc:503] Master 127.6.153.254:34031 was elected leader, sending a full tablet report...
I20240501 14:10:31.019258 26484 ts_tablet_manager.cc:902] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:31.019953 26484 tablet_copy_client.cc:308] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:31.023474 24412 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet 777ba3b6be404d12bb0b219c66da71a6 from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:41349)
I20240501 14:10:31.023600 24412 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet 777ba3b6be404d12bb0b219c66da71a6 from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:41349: session id = 6d832f3f723641778e60b3fd855ff98e-777ba3b6be404d12bb0b219c66da71a6
I20240501 14:10:31.024714 24412 tablet_copy_source_session.cc:205] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.027863 26484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.032864 26484 tablet_copy_client.cc:783] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.033002 26484 tablet_copy_client.cc:647] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.034462 26484 tablet_copy_client.cc:523] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.035761 26484 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:31.040711 26484 log.cc:824] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:31.060185 26484 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.060529 26484 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:31.060809 26484 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.025s user 0.024s sys 0.000s
I20240501 14:10:31.062709 26484 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.062916 26484 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:31.063187 26484 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.064112 26484 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20240501 14:10:31.064833 24412 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-777ba3b6be404d12bb0b219c66da71a6 received from {username='slave'} at 127.6.153.200:41349
I20240501 14:10:31.064939 24412 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-777ba3b6be404d12bb0b219c66da71a6 on tablet 777ba3b6be404d12bb0b219c66da71a6 with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:31.147184 26484 ts_tablet_manager.cc:902] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:31.147656 26484 tablet_copy_client.cc:308] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:31.148138 24412 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet fccb18388fb244bcb1dfd85908c0c118 from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:41349)
I20240501 14:10:31.148248 24412 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet fccb18388fb244bcb1dfd85908c0c118 from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:41349: session id = 6d832f3f723641778e60b3fd855ff98e-fccb18388fb244bcb1dfd85908c0c118
I20240501 14:10:31.149014 24412 tablet_copy_source_session.cc:205] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.149838 26484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.151669 26484 tablet_copy_client.cc:783] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.151814 26484 tablet_copy_client.cc:647] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.152837 26531 ts_tablet_manager.cc:902] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:31.153107 26484 tablet_copy_client.cc:523] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.153659 26531 tablet_copy_client.cc:308] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:46669
I20240501 14:10:31.154234 26484 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:31.158572 24148 tablet_copy_service.cc:140] P 166d7170d98944e88de2eccda9d30545: Received BeginTabletCopySession request for tablet fee07797b5c84ad8aa71d10c1ac24fe1 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:39207)
I20240501 14:10:31.158715 24148 tablet_copy_service.cc:161] P 166d7170d98944e88de2eccda9d30545: Beginning new tablet copy session on tablet fee07797b5c84ad8aa71d10c1ac24fe1 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:39207: session id = c958143102d04279ac25aa56214d0d4b-fee07797b5c84ad8aa71d10c1ac24fe1
I20240501 14:10:31.160104 24148 tablet_copy_source_session.cc:205] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.164034 26531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.165884 26543 ts_tablet_manager.cc:902] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:31.166493 26543 tablet_copy_client.cc:308] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.199:35305
I20240501 14:10:31.170856 24544 tablet_copy_service.cc:140] P 40b69f74de5a447f86c1445f8e84850d: Received BeginTabletCopySession request for tablet 182d85ec9c03441fb974ee50ad36ea8b from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:49285)
I20240501 14:10:31.170959 26531 tablet_copy_client.cc:783] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.170990 24544 tablet_copy_service.cc:161] P 40b69f74de5a447f86c1445f8e84850d: Beginning new tablet copy session on tablet 182d85ec9c03441fb974ee50ad36ea8b from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:49285: session id = 6d832f3f723641778e60b3fd855ff98e-182d85ec9c03441fb974ee50ad36ea8b
I20240501 14:10:31.171133 26531 tablet_copy_client.cc:647] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.172163 24544 tablet_copy_source_session.cc:205] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.173749 26543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.174744 26531 tablet_copy_client.cc:523] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.176486 26531 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:31.176690 26543 tablet_copy_client.cc:783] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.176831 26543 tablet_copy_client.cc:647] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.178014 26543 tablet_copy_client.cc:523] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.178568 26484 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.178804 26484 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:31.178956 26543 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:31.178977 26484 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.025s user 0.016s sys 0.004s
I20240501 14:10:31.179502 26484 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.179632 26484 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:31.179728 26484 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.180433 26484 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 14:10:31.181617 24412 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-fccb18388fb244bcb1dfd85908c0c118 received from {username='slave'} at 127.6.153.200:41349
I20240501 14:10:31.181694 24412 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-fccb18388fb244bcb1dfd85908c0c118 on tablet fccb18388fb244bcb1dfd85908c0c118 with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:31.183599 26531 log.cc:824] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:31.199586 26555 ts_tablet_manager.cc:902] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:31.200525 26555 tablet_copy_client.cc:308] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:38101
I20240501 14:10:31.205755 26543 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.206048 26543 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:31.206218 26543 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.027s user 0.022s sys 0.000s
I20240501 14:10:31.206650 26543 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.206789 26543 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:31.206889 26543 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.207171 26543 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:31.207784 24544 tablet_copy_service.cc:342] P 40b69f74de5a447f86c1445f8e84850d: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-182d85ec9c03441fb974ee50ad36ea8b received from {username='slave'} at 127.6.153.200:49285
I20240501 14:10:31.207888 24544 tablet_copy_service.cc:433] P 40b69f74de5a447f86c1445f8e84850d: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-182d85ec9c03441fb974ee50ad36ea8b on tablet 182d85ec9c03441fb974ee50ad36ea8b with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:31.211104 24280 tablet_copy_service.cc:140] P dbeb6a07fa7a44019ae00c169c414529: Received BeginTabletCopySession request for tablet 158b5efe75ce450abd20149d89157edc from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:33507)
I20240501 14:10:31.211248 24280 tablet_copy_service.cc:161] P dbeb6a07fa7a44019ae00c169c414529: Beginning new tablet copy session on tablet 158b5efe75ce450abd20149d89157edc from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:33507: session id = a308c0ed71f44f57967c97239d1198f7-158b5efe75ce450abd20149d89157edc
I20240501 14:10:31.212589 24280 tablet_copy_source_session.cc:205] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.212683 26531 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.212990 26531 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:31.213300 26531 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.037s user 0.034s sys 0.000s
I20240501 14:10:31.215541 26531 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:31.215759 26531 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:31.215976 26555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158b5efe75ce450abd20149d89157edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.216089 26531 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:31.217062 26531 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20240501 14:10:31.217746 24148 tablet_copy_service.cc:342] P 166d7170d98944e88de2eccda9d30545: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-fee07797b5c84ad8aa71d10c1ac24fe1 received from {username='slave'} at 127.6.153.202:39207
I20240501 14:10:31.217832 24148 tablet_copy_service.cc:433] P 166d7170d98944e88de2eccda9d30545: ending tablet copy session c958143102d04279ac25aa56214d0d4b-fee07797b5c84ad8aa71d10c1ac24fe1 on tablet fee07797b5c84ad8aa71d10c1ac24fe1 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:31.221869 26555 tablet_copy_client.cc:783] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.222050 26555 tablet_copy_client.cc:647] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.223798 26555 tablet_copy_client.cc:523] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.225173 26555 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:31.231179 26555 log.cc:824] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: Log is configured to *not* fsync() on all Append() calls
I20240501 14:10:31.257632 26555 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.257918 26555 tablet_bootstrap.cc:492] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:31.258157 26555 ts_tablet_manager.cc:1367] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.033s user 0.031s sys 0.000s
I20240501 14:10:31.260005 26555 raft_consensus.cc:348] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:31.260211 26555 raft_consensus.cc:729] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:31.260481 26555 consensus_queue.cc:260] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:31.261211 26555 ts_tablet_manager.cc:1398] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20240501 14:10:31.261900 24280 tablet_copy_service.cc:342] P dbeb6a07fa7a44019ae00c169c414529: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-158b5efe75ce450abd20149d89157edc received from {username='slave'} at 127.6.153.201:33507
I20240501 14:10:31.261997 24280 tablet_copy_service.cc:433] P dbeb6a07fa7a44019ae00c169c414529: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-158b5efe75ce450abd20149d89157edc on tablet 158b5efe75ce450abd20149d89157edc with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:31.282150 26555 ts_tablet_manager.cc:902] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427)
I20240501 14:10:31.282567 26555 tablet_copy_client.cc:308] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:34427
I20240501 14:10:31.285778 23884 tablet_copy_service.cc:140] P 2502b9e8098b4bc5af7204835fce444b: Received BeginTabletCopySession request for tablet 8d93a76aca9341088d460780323adbc3 from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:37867)
I20240501 14:10:31.285950 23884 tablet_copy_service.cc:161] P 2502b9e8098b4bc5af7204835fce444b: Beginning new tablet copy session on tablet 8d93a76aca9341088d460780323adbc3 from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:37867: session id = a308c0ed71f44f57967c97239d1198f7-8d93a76aca9341088d460780323adbc3
I20240501 14:10:31.287225 23884 tablet_copy_source_session.cc:205] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.288285 26555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d93a76aca9341088d460780323adbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.289861 26555 tablet_copy_client.cc:783] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.289981 26555 tablet_copy_client.cc:647] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.291173 26555 tablet_copy_client.cc:523] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.292079 26555 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:31.308858 26543 ts_tablet_manager.cc:902] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:31.309259 26543 tablet_copy_client.cc:308] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:44313
I20240501 14:10:31.311283 26555 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.311568 26555 tablet_bootstrap.cc:492] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:31.311738 26555 ts_tablet_manager.cc:1367] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.001s
I20240501 14:10:31.312285 26555 raft_consensus.cc:348] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:31.312433 26555 raft_consensus.cc:729] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:31.312536 26555 consensus_queue.cc:260] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:31.313333 26555 ts_tablet_manager.cc:1398] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 14:10:31.314134 23884 tablet_copy_service.cc:342] P 2502b9e8098b4bc5af7204835fce444b: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-8d93a76aca9341088d460780323adbc3 received from {username='slave'} at 127.6.153.201:37867
I20240501 14:10:31.314237 23884 tablet_copy_service.cc:433] P 2502b9e8098b4bc5af7204835fce444b: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-8d93a76aca9341088d460780323adbc3 on tablet 8d93a76aca9341088d460780323adbc3 with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:31.314827 24016 tablet_copy_service.cc:140] P 8ebe0a098b0f4ed4861a5605717fecbd: Received BeginTabletCopySession request for tablet be60ec7c4b0a46c28e0b23b6facae47e from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:49861)
I20240501 14:10:31.314985 24016 tablet_copy_service.cc:161] P 8ebe0a098b0f4ed4861a5605717fecbd: Beginning new tablet copy session on tablet be60ec7c4b0a46c28e0b23b6facae47e from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:49861: session id = 6d832f3f723641778e60b3fd855ff98e-be60ec7c4b0a46c28e0b23b6facae47e
I20240501 14:10:31.316429 24016 tablet_copy_source_session.cc:205] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.318033 26543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be60ec7c4b0a46c28e0b23b6facae47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.319985 26543 tablet_copy_client.cc:783] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.320137 26543 tablet_copy_client.cc:647] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.321391 26543 tablet_copy_client.cc:523] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.322372 26543 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:31.333794 26484 ts_tablet_manager.cc:902] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:31.334355 26484 tablet_copy_client.cc:308] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:44313
I20240501 14:10:31.334816 24016 tablet_copy_service.cc:140] P 8ebe0a098b0f4ed4861a5605717fecbd: Received BeginTabletCopySession request for tablet 207a860a62e34c619ecec37f1eacaacd from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:49861)
I20240501 14:10:31.334908 24016 tablet_copy_service.cc:161] P 8ebe0a098b0f4ed4861a5605717fecbd: Beginning new tablet copy session on tablet 207a860a62e34c619ecec37f1eacaacd from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:49861: session id = 6d832f3f723641778e60b3fd855ff98e-207a860a62e34c619ecec37f1eacaacd
I20240501 14:10:31.335666 24016 tablet_copy_source_session.cc:205] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.337011 26484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 207a860a62e34c619ecec37f1eacaacd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.338567 26484 tablet_copy_client.cc:783] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.338707 26484 tablet_copy_client.cc:647] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.340006 26484 tablet_copy_client.cc:523] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.342415 26543 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.342643 26543 tablet_bootstrap.cc:492] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:31.342792 26543 ts_tablet_manager.cc:1367] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20240501 14:10:31.342993 26484 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:31.343183 26543 raft_consensus.cc:348] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.343315 26543 raft_consensus.cc:729] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:31.343444 26543 consensus_queue.cc:260] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.344095 26543 ts_tablet_manager.cc:1398] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.001s sys 0.004s
I20240501 14:10:31.344653 24016 tablet_copy_service.cc:342] P 8ebe0a098b0f4ed4861a5605717fecbd: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-be60ec7c4b0a46c28e0b23b6facae47e received from {username='slave'} at 127.6.153.200:49861
I20240501 14:10:31.344749 24016 tablet_copy_service.cc:433] P 8ebe0a098b0f4ed4861a5605717fecbd: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-be60ec7c4b0a46c28e0b23b6facae47e on tablet be60ec7c4b0a46c28e0b23b6facae47e with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:31.350350 26531 ts_tablet_manager.cc:902] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:31.350729 26531 tablet_copy_client.cc:308] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:31.354462 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet 697668f5caf34e63a9e5953c180959ee from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:44465)
I20240501 14:10:31.354585 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet 697668f5caf34e63a9e5953c180959ee from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:44465: session id = c958143102d04279ac25aa56214d0d4b-697668f5caf34e63a9e5953c180959ee
I20240501 14:10:31.355783 23752 tablet_copy_source_session.cc:205] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.357342 26531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.359310 26531 tablet_copy_client.cc:783] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.359477 26531 tablet_copy_client.cc:647] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.360706 26531 tablet_copy_client.cc:523] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.361869 26531 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:31.364663 26484 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.364991 26484 tablet_bootstrap.cc:492] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:31.365156 26484 ts_tablet_manager.cc:1367] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.023s user 0.021s sys 0.000s
I20240501 14:10:31.365533 26484 raft_consensus.cc:348] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.365639 26484 raft_consensus.cc:729] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:31.365718 26484 consensus_queue.cc:260] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:31.366426 26484 ts_tablet_manager.cc:1398] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:31.366876 24016 tablet_copy_service.cc:342] P 8ebe0a098b0f4ed4861a5605717fecbd: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-207a860a62e34c619ecec37f1eacaacd received from {username='slave'} at 127.6.153.200:49861
I20240501 14:10:31.366971 24016 tablet_copy_service.cc:433] P 8ebe0a098b0f4ed4861a5605717fecbd: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-207a860a62e34c619ecec37f1eacaacd on tablet 207a860a62e34c619ecec37f1eacaacd with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:31.374858 26591 ts_tablet_manager.cc:902] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:31.375411 26591 tablet_copy_client.cc:308] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:31.378566 24412 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet a50dc90c18174aa289c75056c4c461b0 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:59269)
I20240501 14:10:31.378654 24412 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet a50dc90c18174aa289c75056c4c461b0 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:59269: session id = c958143102d04279ac25aa56214d0d4b-a50dc90c18174aa289c75056c4c461b0
I20240501 14:10:31.379308 24412 tablet_copy_source_session.cc:205] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:31.380045 26591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:31.381983 26591 tablet_copy_client.cc:783] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:31.382131 26591 tablet_copy_client.cc:647] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:31.382536 26531 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.382745 26531 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:31.382896 26531 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.001s
I20240501 14:10:31.383131 26591 tablet_copy_client.cc:523] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:31.383375 26531 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:31.383486 26531 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:31.383582 26531 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:31.383841 26531 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.001s sys 0.003s
I20240501 14:10:31.384413 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-697668f5caf34e63a9e5953c180959ee received from {username='slave'} at 127.6.153.202:44465
I20240501 14:10:31.384505 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session c958143102d04279ac25aa56214d0d4b-697668f5caf34e63a9e5953c180959ee on tablet 697668f5caf34e63a9e5953c180959ee with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:31.385797 26591 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:31.405946 26591 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:31.406273 26591 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:31.406497 26591 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.021s user 0.020s sys 0.000s
I20240501 14:10:31.406934 26591 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:31.407101 26591 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:31.407210 26591 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:31.407608 26591 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:31.408670 24412 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-a50dc90c18174aa289c75056c4c461b0 received from {username='slave'} at 127.6.153.202:59269
I20240501 14:10:31.408749 24412 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session c958143102d04279ac25aa56214d0d4b-a50dc90c18174aa289c75056c4c461b0 on tablet a50dc90c18174aa289c75056c4c461b0 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:31.596953 26072 raft_consensus.cc:1203] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.625593 25941 raft_consensus.cc:1203] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.683708 25808 raft_consensus.cc:1203] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.705785 26072 raft_consensus.cc:1203] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.771206 26072 raft_consensus.cc:1203] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.793906 25808 raft_consensus.cc:1203] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.815140 26072 raft_consensus.cc:1203] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.861730 25941 raft_consensus.cc:1203] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.872442 25941 raft_consensus.cc:1203] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:31.910065 26072 raft_consensus.cc:1203] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:32.012391 26648 raft_consensus.cc:1050] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:32.012892 26648 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:32.014763 24524 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.014776 24258 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.014843 23731 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.015226 26758 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.015486 26760 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.015719 24390 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.015785 26759 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.015934 25941 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.016139 26759 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.016803 23994 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.016886 23862 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.017155 26760 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.017374 26759 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.018467 26759 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.019667 26759 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.020148 23994 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.020727 25941 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.021091 23731 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.023324 23588 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:32.023417 24390 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.023483 24258 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.023751 23862 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.024567 24524 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.029616 24127 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:32.031247 24258 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.031555 24524 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.031555 23994 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.031626 26647 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.031883 26648 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.032085 26647 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.033239 23731 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.033468 25941 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.033562 26761 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.033704 23862 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.033890 26761 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.034093 26648 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.034240 26758 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.034425 23994 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.035207 25941 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.035621 24258 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.036839 23586 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.037609 23862 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.038411 24524 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.040094 23730 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.041078 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fee07797b5c84ad8aa71d10c1ac24fe1 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.041196 26301 raft_consensus.cc:1050] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:32.041627 26301 consensus_queue.cc:237] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.043291 24371 tablet_service.cc:1514] Processing DeleteTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 with delete_type TABLET_DATA_TOMBSTONED (TS 7e6e9293a4614dabb5ad89fe3f4d5d04 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44666
I20240501 14:10:32.043649 24523 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.043848 26778 tablet_replica.cc:330] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: stopping tablet replica
I20240501 14:10:32.044170 23864 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.044183 26778 raft_consensus.cc:2229] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.044299 26072 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.044533 26778 raft_consensus.cc:2258] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.044729 26301 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.044968 24258 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.045042 26301 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.045917 26272 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.046159 26302 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.046716 26778 ts_tablet_manager.cc:1875] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.047202 23730 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.047608 26301 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.049432 26778 ts_tablet_manager.cc:1888] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.049506 26778 log.cc:1196] T fee07797b5c84ad8aa71d10c1ac24fe1 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/wals/fee07797b5c84ad8aa71d10c1ac24fe1
I20240501 14:10:32.050290 23574 catalog_manager.cc:4830] TS 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897): tablet fee07797b5c84ad8aa71d10c1ac24fe1 (table rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]) successfully deleted
I20240501 14:10:32.050670 26301 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.050983 24127 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.051378 23994 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.051391 26272 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.051759 26272 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.052469 24258 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.052634 24523 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.053114 24127 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.054206 26072 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.055187 23590 catalog_manager.cc:5484] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:32.056654 23730 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.057547 23862 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.055233 23994 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.191082 26584 raft_consensus.cc:1050] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:32.191700 26584 consensus_queue.cc:237] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.193962 24260 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.194067 23730 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.194396 24390 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.194509 24523 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.194541 24127 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.194725 26819 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.195209 26365 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.195523 23862 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.195556 26365 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.195698 25492 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.196365 25492 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.196493 26072 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.197100 26820 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.198702 25492 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.198855 26338 raft_consensus.cc:1050] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:32.199447 26338 consensus_queue.cc:237] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:32.200981 25492 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.201546 24523 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.201615 23864 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.201956 26675 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.202107 24392 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.202275 26675 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.202144 26072 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.202383 23730 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.203152 25808 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.204140 23581 catalog_manager.cc:5484] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:32.204206 24126 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.204406 24260 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.205885 23994 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.206302 26338 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.206691 26338 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.206569 24523 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.206734 23730 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.207728 26338 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.210393 26836 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.213476 23862 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.213841 26675 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.214512 24392 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.214684 24127 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.215512 26338 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.215868 25808 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.216686 24127 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.219365 23586 catalog_manager.cc:5484] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 reported cstate change: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.220458 24392 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.221575 24524 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.222842 23994 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.222858 23862 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.224072 23731 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.228487 23994 consensus_queue.cc:237] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.230579 24127 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.230634 23731 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.230939 26072 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.230978 26817 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.231374 23862 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.231819 26818 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.231972 26817 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.232316 24524 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.232308 26821 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.232813 24259 raft_consensus.cc:1261] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.233219 26818 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.233475 26818 consensus_queue.cc:1035] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.234489 26821 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.235035 24127 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.235100 23731 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.237208 23577 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 207a860a62e34c619ecec37f1eacaacd with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.237073 26072 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.237704 23584 catalog_manager.cc:5484] T 207a860a62e34c619ecec37f1eacaacd P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.237957 24259 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.240274 24260 consensus_queue.cc:237] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:32.241772 24524 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.242228 24523 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.242919 25808 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.242813 23862 raft_consensus.cc:2932] T 207a860a62e34c619ecec37f1eacaacd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.243364 26840 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.244896 26302 raft_consensus.cc:1050] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:32.245132 24392 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.245388 26302 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:32.246588 26839 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.246881 26839 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.247716 23994 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.248023 26840 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.248351 23862 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.248687 26840 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.249713 25941 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.250208 26774 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.250346 23730 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.250725 26774 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.252175 24127 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.252175 24523 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.252424 23994 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.252822 23862 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.252810 26776 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.253360 24258 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.253530 26565 raft_consensus.cc:1050] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:32.253531 23731 raft_consensus.cc:1261] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.253706 26839 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.254030 26776 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.254025 26565 consensus_queue.cc:237] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.254959 26839 consensus_queue.cc:1035] T 158b5efe75ce450abd20149d89157edc P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.255807 24371 tablet_service.cc:1514] Processing DeleteTablet for tablet 207a860a62e34c619ecec37f1eacaacd with delete_type TABLET_DATA_TOMBSTONED (TS 7e6e9293a4614dabb5ad89fe3f4d5d04 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44666
I20240501 14:10:32.256048 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 158b5efe75ce450abd20149d89157edc with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.256505 26778 tablet_replica.cc:330] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: stopping tablet replica
I20240501 14:10:32.256637 26778 raft_consensus.cc:2229] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.256824 26778 raft_consensus.cc:2258] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.256803 23731 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.256899 25808 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.257804 23730 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.258077 24127 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.258256 23864 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.258329 26072 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.258832 26774 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.259074 24390 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.259651 26778 ts_tablet_manager.cc:1875] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.259680 24258 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.259943 23582 catalog_manager.cc:5484] T 158b5efe75ce450abd20149d89157edc P a308c0ed71f44f57967c97239d1198f7 reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.260063 23995 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.259081 26272 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20240501 14:10:32.260764 26301 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.261304 24392 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.261929 26389 raft_consensus.cc:1050] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:32.262451 26389 consensus_queue.cc:237] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:32.262681 23864 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.263847 26778 ts_tablet_manager.cc:1888] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.264271 23731 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.264552 23994 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.264633 26778 log.cc:1196] T 207a860a62e34c619ecec37f1eacaacd P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/wals/207a860a62e34c619ecec37f1eacaacd
I20240501 14:10:32.265106 24524 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.265481 26704 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.265661 23574 catalog_manager.cc:4830] TS 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897): tablet 207a860a62e34c619ecec37f1eacaacd (table rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]) successfully deleted
I20240501 14:10:32.265808 26704 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.266103 26704 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.267119 26858 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.267566 24258 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.268675 25808 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.269148 26703 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.269079 23994 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.270875 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 158b5efe75ce450abd20149d89157edc with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:32.271003 26774 raft_consensus.cc:1050] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:32.270951 25941 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.271456 26774 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.271613 24522 raft_consensus.cc:2932] T 158b5efe75ce450abd20149d89157edc P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.271762 24126 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.273115 23995 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.273527 24260 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.273571 23730 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.274011 26777 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.274106 26072 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.274657 26302 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.274726 23584 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.275360 24522 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.276711 26302 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.277359 23995 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.277676 26302 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.277824 23864 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.277820 24390 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.277956 26302 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.278088 24127 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.278210 24258 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.278815 26703 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.279129 26703 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.279477 26703 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.280758 26777 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.283071 26704 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.283380 24390 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.284531 23732 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.284933 24126 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.285566 23995 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.285918 26858 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.286319 26882 tablet_replica.cc:330] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:32.286276 24260 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.286620 26882 raft_consensus.cc:2229] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.287971 23588 catalog_manager.cc:5484] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.288372 26882 raft_consensus.cc:2258] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.288646 24521 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.289597 26882 ts_tablet_manager.cc:1875] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.290838 26302 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.291594 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.292055 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.292466 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.292484 26072 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.292641 24126 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.292862 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.293231 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.294793 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.294778 23589 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e reported cstate change: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.295344 24521 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.297128 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.297560 23732 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.301149 23995 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.301685 23731 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.304137 26882 ts_tablet_manager.cc:1888] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.304384 26882 log.cc:1196] T 158b5efe75ce450abd20149d89157edc P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/158b5efe75ce450abd20149d89157edc
I20240501 14:10:32.305193 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet 158b5efe75ce450abd20149d89157edc (table rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]) successfully deleted
I20240501 14:10:32.307440 24524 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.308182 25808 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.308496 23863 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.310273 26896 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.312426 23863 consensus_queue.cc:237] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:32.312960 23588 catalog_manager.cc:5484] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.312956 23864 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.314810 24259 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.315191 24390 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.315238 26389 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.315639 26389 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.315958 23730 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.316411 25808 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.316738 26704 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.317289 26704 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.317405 24524 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.317687 23995 raft_consensus.cc:1261] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.317914 26704 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.318054 26389 consensus_queue.cc:1035] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.319679 24390 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.319713 23994 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.320050 26072 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.321092 26704 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.322150 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.322525 24128 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.322481 26858 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.323581 26858 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.323716 23864 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.324009 26858 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.324137 24258 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.324358 24524 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.324683 26858 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.326129 24390 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.326941 23994 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.327378 23577 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8d93a76aca9341088d460780323adbc3 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.327764 26072 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.328132 26775 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.328415 26775 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.328612 23995 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.328823 23730 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.329250 23587 catalog_manager.cc:5484] T 8d93a76aca9341088d460780323adbc3 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:32.329622 24524 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.331010 25808 raft_consensus.cc:2932] T 8d93a76aca9341088d460780323adbc3 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:32.331781 26858 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.332482 24260 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.332945 23731 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.334265 26072 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.334319 24392 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.335898 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d93a76aca9341088d460780323adbc3 with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:32.336613 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fccb18388fb244bcb1dfd85908c0c118 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.336763 24127 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.337064 23864 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.340034 23581 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.340461 24524 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.342341 23994 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.343124 26882 tablet_replica.cc:330] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:32.343425 26882 raft_consensus.cc:2229] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.343878 23731 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.344841 24126 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.345886 26882 pending_rounds.cc:62] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: Trying to abort 1 pending ops.
I20240501 14:10:32.346041 26882 pending_rounds.cc:68] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: Aborting op as it isn't in flight: id { term: 1 index: 6 } timestamp: 7022889501745451008 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "8d93a76aca9341088d460780323adbc3" old_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } } new_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } } }
I20240501 14:10:32.346345 26882 raft_consensus.cc:2894] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Aborting config change with OpId 1.6: Aborted: Op aborted
I20240501 14:10:32.346472 26882 raft_consensus.cc:2866] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240501 14:10:32.346571 26882 raft_consensus.cc:2258] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.346643 24239 tablet_service.cc:1514] Processing DeleteTablet for tablet fccb18388fb244bcb1dfd85908c0c118 with delete_type TABLET_DATA_TOMBSTONED (TS dbeb6a07fa7a44019ae00c169c414529 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:51508
I20240501 14:10:32.347141 23864 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.349705 24524 consensus_queue.cc:237] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.351516 26882 ts_tablet_manager.cc:1875] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.351554 26072 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.351859 24390 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.352058 26866 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.352576 24390 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:32.353452 26882 ts_tablet_manager.cc:1888] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.353521 26882 log.cc:1196] T 8d93a76aca9341088d460780323adbc3 P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/8d93a76aca9341088d460780323adbc3
I20240501 14:10:32.353533 23731 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.354169 26909 tablet_replica.cc:330] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: stopping tablet replica
I20240501 14:10:32.354461 24524 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.354463 23994 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.354478 25941 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.354794 24260 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.354768 26774 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.354923 23864 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.354916 26858 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.355183 26301 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.355435 26858 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.355587 26884 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.356271 23864 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.356474 23994 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.357126 23731 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.357637 26858 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.357666 26866 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.358023 26775 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.358532 23864 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.358706 25941 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.359230 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.359540 26565 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.360072 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a50dc90c18174aa289c75056c4c461b0 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.360220 24260 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.361083 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet 8d93a76aca9341088d460780323adbc3 (table rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]) successfully deleted
I20240501 14:10:32.360991 23995 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.362277 23586 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:32.362917 26864 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.362821 23730 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.365255 24521 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.366706 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet a50dc90c18174aa289c75056c4c461b0 with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:32.367476 26882 tablet_replica.cc:330] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:32.367646 26882 raft_consensus.cc:2229] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.367981 26882 raft_consensus.cc:2258] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.368326 26909 raft_consensus.cc:2229] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.368232 24390 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.368618 26909 pending_rounds.cc:62] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: Trying to abort 1 pending ops.
I20240501 14:10:32.368716 26909 pending_rounds.cc:68] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: Aborting op as it isn't in flight: id { term: 1 index: 6 } timestamp: 7022889501782421504 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "fccb18388fb244bcb1dfd85908c0c118" old_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } } new_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } } }
I20240501 14:10:32.368964 26909 raft_consensus.cc:2894] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Aborting config change with OpId 1.6: Aborted: Op aborted
I20240501 14:10:32.369068 26909 raft_consensus.cc:2866] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240501 14:10:32.369158 26909 raft_consensus.cc:2258] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.370391 23864 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.369531 26865 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.371443 26909 ts_tablet_manager.cc:1875] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.372283 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 182d85ec9c03441fb974ee50ad36ea8b with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.372575 23731 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.372785 24126 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.372822 26072 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.373150 26864 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.373081 23586 catalog_manager.cc:5484] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New cstate: current_term: 1 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:32.374033 24126 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.375597 23994 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.376216 26882 ts_tablet_manager.cc:1875] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.377202 24239 tablet_service.cc:1514] Processing DeleteTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b with delete_type TABLET_DATA_TOMBSTONED (TS dbeb6a07fa7a44019ae00c169c414529 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:51508
I20240501 14:10:32.378530 26882 ts_tablet_manager.cc:1888] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.378600 26882 log.cc:1196] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/a50dc90c18174aa289c75056c4c461b0
I20240501 14:10:32.379149 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet a50dc90c18174aa289c75056c4c461b0 (table rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]) successfully deleted
I20240501 14:10:32.379469 26909 ts_tablet_manager.cc:1888] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.379587 26909 log.cc:1196] T fccb18388fb244bcb1dfd85908c0c118 P dbeb6a07fa7a44019ae00c169c414529: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/fccb18388fb244bcb1dfd85908c0c118
I20240501 14:10:32.380115 26909 tablet_replica.cc:330] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: stopping tablet replica
I20240501 14:10:32.380137 23575 catalog_manager.cc:4830] TS dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): tablet fccb18388fb244bcb1dfd85908c0c118 (table rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]) successfully deleted
I20240501 14:10:32.380234 26909 raft_consensus.cc:2229] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.380431 26909 raft_consensus.cc:2258] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.380986 26909 ts_tablet_manager.cc:1875] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.382467 26909 ts_tablet_manager.cc:1888] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.382525 26909 log.cc:1196] T 182d85ec9c03441fb974ee50ad36ea8b P dbeb6a07fa7a44019ae00c169c414529: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/182d85ec9c03441fb974ee50ad36ea8b
I20240501 14:10:32.382927 23575 catalog_manager.cc:4830] TS dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): tablet 182d85ec9c03441fb974ee50ad36ea8b (table rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]) successfully deleted
I20240501 14:10:32.383512 26818 raft_consensus.cc:1050] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:32.383919 26818 consensus_queue.cc:237] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.385391 26072 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.385887 26821 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.386025 23864 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.386447 24390 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.386430 26817 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.386750 24524 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.386948 26817 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.387583 24260 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.387766 26817 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.387936 26820 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.388630 26706 raft_consensus.cc:1050] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:32.389153 26706 consensus_queue.cc:237] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:32.390594 26817 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.390878 24524 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.391214 24258 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.391191 26398 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.391438 24127 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.391541 26398 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.392006 23994 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.392278 23862 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.392346 26072 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.392956 24390 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.393332 24260 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.393669 24524 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.393898 25941 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.394331 26706 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.394629 26706 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.394767 23590 catalog_manager.cc:5484] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:32.394941 26706 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.395684 24390 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.395823 26706 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.398581 23731 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.399143 26918 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.399369 26820 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.400681 26918 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.401435 23731 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.401955 25941 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.402431 24390 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.402762 23862 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.403666 23994 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.404137 23590 catalog_manager.cc:5484] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.406210 24126 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:32.406540 26820 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:32.406423 24127 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.407076 24258 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.407548 24126 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.416558 24521 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.417039 23864 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.420161 23731 consensus_queue.cc:237] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:32.421698 25941 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.421757 24521 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.422119 24390 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.422163 26398 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.422446 26398 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.422768 26918 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.423158 23994 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.423722 24126 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.424005 23864 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.424481 26918 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.424660 26706 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.424947 26706 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.425676 26918 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.426911 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 697668f5caf34e63a9e5953c180959ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.427248 23994 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.427496 25941 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.428422 23589 catalog_manager.cc:5484] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:32.429034 23864 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.429132 24390 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.431100 24126 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.431658 24521 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:32.432241 24239 tablet_service.cc:1514] Processing DeleteTablet for tablet 697668f5caf34e63a9e5953c180959ee with delete_type TABLET_DATA_TOMBSTONED (TS dbeb6a07fa7a44019ae00c169c414529 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:51508
I20240501 14:10:32.432472 26909 tablet_replica.cc:330] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: stopping tablet replica
I20240501 14:10:32.432600 26909 raft_consensus.cc:2229] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.432744 26909 raft_consensus.cc:2258] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.434113 26909 ts_tablet_manager.cc:1875] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.435976 26909 ts_tablet_manager.cc:1888] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.436038 26909 log.cc:1196] T 697668f5caf34e63a9e5953c180959ee P dbeb6a07fa7a44019ae00c169c414529: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/697668f5caf34e63a9e5953c180959ee
I20240501 14:10:32.436400 23575 catalog_manager.cc:4830] TS dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): tablet 697668f5caf34e63a9e5953c180959ee (table rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]) successfully deleted
I20240501 14:10:32.647505 24390 consensus_queue.cc:237] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.649020 26072 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.649266 23730 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.649407 26774 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.649713 24258 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.650274 24126 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.650311 26884 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.650610 26884 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.651185 26858 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.652320 23864 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.652447 23994 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.652732 26774 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.652895 26858 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.653295 26858 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.653834 24126 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.654965 24258 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.655350 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 777ba3b6be404d12bb0b219c66da71a6 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.656622 26072 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.656934 23584 catalog_manager.cc:5484] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 6 to 7, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.657238 23864 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.658011 23994 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.660073 24504 tablet_service.cc:1514] Processing DeleteTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 with delete_type TABLET_DATA_TOMBSTONED (TS 40b69f74de5a447f86c1445f8e84850d not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58450
I20240501 14:10:32.660646 26971 tablet_replica.cc:330] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: stopping tablet replica
I20240501 14:10:32.661006 26971 raft_consensus.cc:2229] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.661296 26971 raft_consensus.cc:2258] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.661985 23730 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.662467 26971 ts_tablet_manager.cc:1875] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.664645 26971 ts_tablet_manager.cc:1888] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.664726 26971 log.cc:1196] T 777ba3b6be404d12bb0b219c66da71a6 P 40b69f74de5a447f86c1445f8e84850d: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/wals/777ba3b6be404d12bb0b219c66da71a6
I20240501 14:10:32.665313 23576 catalog_manager.cc:4830] TS 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305): tablet 777ba3b6be404d12bb0b219c66da71a6 (table rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]) successfully deleted
I20240501 14:10:32.818984 23994 consensus_queue.cc:237] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:32.820480 24390 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.820564 24524 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.820865 23730 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.820860 26819 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.821013 26820 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.821261 24258 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.821599 26820 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.821800 23864 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.822355 26820 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.822387 24126 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.822750 26820 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.822937 26819 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.824648 26820 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.824963 23862 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.824975 24390 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.828236 24524 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.828300 24126 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.828863 23584 catalog_manager.cc:5484] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.831928 23730 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.833442 24258 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.836707 23730 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:32.838287 23994 consensus_queue.cc:237] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:32.840219 24258 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.840452 24260 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.840826 24390 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.840806 26365 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.841454 23862 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.841619 24126 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.841912 23731 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.842224 24524 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.842248 26819 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.842252 26917 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.842540 26817 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.842705 25492 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.842906 26821 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.843902 26819 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.845932 26819 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.846537 24524 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.846899 26920 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.847517 24392 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.847849 24126 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.848095 26918 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.848150 24259 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.848300 26919 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 14:10:32.848660 23929 consensus_peers.cc:485] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb091eaf92de4f9bbf110f489a9f21bd. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:32.849437 24127 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.850172 23864 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.850245 23994 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.850605 26920 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.850757 26917 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.850960 23731 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.851613 24521 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.852952 23589 catalog_manager.cc:5484] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.853732 26919 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.855255 24392 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.855387 24127 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.855409 23862 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.855605 24390 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.858333 23588 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.858640 24260 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.859987 23577 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet bb091eaf92de4f9bbf110f489a9f21bd with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:32.862499 23994 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.863054 24524 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.863363 23864 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.867734 23731 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:32.869350 23864 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.869350 23994 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.869724 24127 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.869766 26918 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.870095 26918 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.870396 26918 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.873414 24390 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.873740 26919 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.873785 24260 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.873566 24524 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.874099 26919 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.874357 26917 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.875635 26918 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.876003 24127 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:32.875924 24260 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
W20240501 14:10:32.876125 23665 consensus_peers.cc:485] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d25ef07189ac43749efc549299dc89f8. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:32.877740 24521 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.877872 23575 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d25ef07189ac43749efc549299dc89f8 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:32.878240 23995 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.878412 26761 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.878520 23584 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.879017 24127 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.879841 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.880322 26648 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.880915 23864 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.881283 26761 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.882141 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.882369 24390 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.882592 26761 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.884753 24524 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.885605 26761 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.887442 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.887477 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.888325 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.888720 26122 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.888998 26122 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.889097 23864 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.889549 23584 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.889719 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.890267 23994 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.891515 23731 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.895112 24127 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:32.895599 23864 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.898197 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.898262 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.898566 26122 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.899127 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.899500 24524 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.899926 26122 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.900143 27009 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.900413 27008 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.900908 23864 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.901072 24521 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.901271 26761 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.902747 27008 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.903707 24390 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.904698 27008 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.905473 24390 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.905493 23994 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.906929 23576 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet de25122a01e845d2b35e4a1023789d3e with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:32.907291 23590 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:32.907406 23731 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.908196 23864 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.909590 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
W20240501 14:10:32.917738 24061 consensus_peers.cc:485] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de25122a01e845d2b35e4a1023789d3e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:32.919660 24521 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.928439 23731 consensus_queue.cc:237] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:32.929934 23994 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.929944 23864 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.930039 24260 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.930465 26921 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.930650 26920 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.930742 24521 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.930971 26917 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.931197 26920 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.931255 24127 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.931671 26921 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.933881 26706 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.934213 24260 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.934422 24521 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.934835 23864 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.936450 23994 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.937055 24127 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.937145 23586 catalog_manager.cc:5484] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.938591 24390 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.938982 26920 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.940238 24390 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:32.940807 23731 consensus_queue.cc:237] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:32.942683 24260 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.943162 26919 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 14:10:32.943499 23665 consensus_peers.cc:485] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba3621a3f5df43eca96826da75fc31d0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:32.943615 23864 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.944062 26918 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.944221 24521 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.944610 24127 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.944609 26921 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.945518 24392 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.945982 26918 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.946204 26921 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.947046 23994 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.947420 26921 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.948097 26919 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:32.948782 23864 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:32.949904 23575 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ba3621a3f5df43eca96826da75fc31d0 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:32.950212 24260 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:32.951889 23994 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:32.952129 24392 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:32.952450 23586 catalog_manager.cc:5484] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:32.954167 24127 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:32.954244 24521 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:32.958035 23994 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:32.959426 23995 consensus_queue.cc:237] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:32.960776 24260 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.961050 26072 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.961167 24524 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.961228 26821 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.961546 24521 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.961599 24126 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.961870 26819 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.961903 24127 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.961966 23731 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.962003 24259 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.962849 26819 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.962931 26821 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.963276 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.963609 26818 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.963644 26819 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.963892 26818 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.963889 26819 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.964376 23864 raft_consensus.cc:1261] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:32.964529 26821 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.964707 26817 consensus_queue.cc:1035] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:32.965090 25492 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.965339 23864 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.965678 26819 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.966172 23731 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.966869 23577 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet be60ec7c4b0a46c28e0b23b6facae47e with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:32.968096 26072 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.968380 23731 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.968684 24126 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.969108 23581 catalog_manager.cc:5484] T be60ec7c4b0a46c28e0b23b6facae47e P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.967168 23864 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.970614 24258 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.972362 24371 tablet_service.cc:1514] Processing DeleteTablet for tablet be60ec7c4b0a46c28e0b23b6facae47e with delete_type TABLET_DATA_TOMBSTONED (TS 7e6e9293a4614dabb5ad89fe3f4d5d04 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44666
I20240501 14:10:32.972923 26820 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.973160 27043 tablet_replica.cc:330] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: stopping tablet replica
I20240501 14:10:32.973326 27043 raft_consensus.cc:2229] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:32.973670 27043 raft_consensus.cc:2258] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:32.974476 27043 ts_tablet_manager.cc:1875] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:32.975131 24521 raft_consensus.cc:2932] T be60ec7c4b0a46c28e0b23b6facae47e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:32.976099 27043 ts_tablet_manager.cc:1888] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:32.976162 27043 log.cc:1196] T be60ec7c4b0a46c28e0b23b6facae47e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/wals/be60ec7c4b0a46c28e0b23b6facae47e
I20240501 14:10:32.976322 26817 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.976765 23574 catalog_manager.cc:4830] TS 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897): tablet be60ec7c4b0a46c28e0b23b6facae47e (table rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]) successfully deleted
I20240501 14:10:32.978816 24258 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.978808 23731 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.979717 24521 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.981173 23584 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.981475 24127 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.985091 23994 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:32.987018 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.987602 24127 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.987833 23864 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 14:10:32.987867 23929 consensus_peers.cc:485] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fc754b4d0f144c4ba0bd9a1b77f7bb4. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:32.987864 24521 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.987960 26820 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.988025 23862 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.988193 26820 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.988276 25492 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.988620 24392 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:32.988754 26820 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.989253 23731 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.989637 26820 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.990257 24258 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:32.990609 26820 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:32.991744 24260 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:32.992645 26820 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.992995 24392 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.993695 23730 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.993841 24390 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.994179 26972 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.994737 24127 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.994876 26972 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.995107 23577 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:32.995285 24521 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.995889 23587 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:32.996608 23994 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.997097 23862 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.997499 26972 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.998062 24521 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:32.998620 26972 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:32.999701 23864 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:33.000290 24127 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:33.000566 23731 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:33.000713 26972 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:33.001919 27052 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:33.002154 24260 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:33.003291 26838 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:33.005075 24392 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:33.005100 23730 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:33.007009 23862 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:33.008037 23590 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:33.009193 23994 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:33.010277 24521 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:33.014704 24258 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:33.015074 24127 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:33.016849 23730 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:33.017706 24392 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:33.018028 23862 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:33.018054 26838 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:33.018473 26838 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:33.018541 24127 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:33.018648 24521 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:33.018985 26973 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:33.019088 26972 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:33.019196 26838 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:33.021262 26972 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:33.021991 23994 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:33.021884 23862 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:33.022436 26838 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:33.023419 23994 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:33.024441 23581 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:33.024434 24392 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:33.026290 23730 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:33.027994 23575 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 34a278486c8b447d8dc4c3241743de60 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:33.028009 24521 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
W20240501 14:10:33.028312 24193 consensus_peers.cc:485] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34a278486c8b447d8dc4c3241743de60. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:33.028535 24127 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:33.370208 27135 ts_tablet_manager.cc:902] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:33.370640 27135 tablet_copy_client.cc:308] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:44313
I20240501 14:10:33.373723 24016 tablet_copy_service.cc:140] P 8ebe0a098b0f4ed4861a5605717fecbd: Received BeginTabletCopySession request for tablet bb091eaf92de4f9bbf110f489a9f21bd from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:51351)
I20240501 14:10:33.373818 24016 tablet_copy_service.cc:161] P 8ebe0a098b0f4ed4861a5605717fecbd: Beginning new tablet copy session on tablet bb091eaf92de4f9bbf110f489a9f21bd from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:51351: session id = a308c0ed71f44f57967c97239d1198f7-bb091eaf92de4f9bbf110f489a9f21bd
I20240501 14:10:33.374550 24016 tablet_copy_source_session.cc:205] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:33.375296 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb091eaf92de4f9bbf110f489a9f21bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:33.377259 27135 tablet_copy_client.cc:783] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:33.377388 27135 tablet_copy_client.cc:647] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:33.378332 27135 tablet_copy_client.cc:523] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:33.379221 27135 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:33.396225 27135 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:33.396435 27135 tablet_bootstrap.cc:492] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:33.396570 27135 ts_tablet_manager.cc:1367] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.001s
I20240501 14:10:33.396888 27135 raft_consensus.cc:348] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:33.396994 27135 raft_consensus.cc:729] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:33.397087 27135 consensus_queue.cc:260] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:33.397404 27135 ts_tablet_manager.cc:1398] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:10:33.397787 24016 tablet_copy_service.cc:342] P 8ebe0a098b0f4ed4861a5605717fecbd: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-bb091eaf92de4f9bbf110f489a9f21bd received from {username='slave'} at 127.6.153.201:51351
I20240501 14:10:33.397861 24016 tablet_copy_service.cc:433] P 8ebe0a098b0f4ed4861a5605717fecbd: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-bb091eaf92de4f9bbf110f489a9f21bd on tablet bb091eaf92de4f9bbf110f489a9f21bd with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:33.406307 27144 ts_tablet_manager.cc:902] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:33.406698 27144 tablet_copy_client.cc:308] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:33.407135 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet ba3621a3f5df43eca96826da75fc31d0 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:44465)
I20240501 14:10:33.407235 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet ba3621a3f5df43eca96826da75fc31d0 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:44465: session id = c958143102d04279ac25aa56214d0d4b-ba3621a3f5df43eca96826da75fc31d0
I20240501 14:10:33.408064 23752 tablet_copy_source_session.cc:205] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:33.408784 27144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:33.410753 27144 tablet_copy_client.cc:783] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:33.410897 27144 tablet_copy_client.cc:647] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:33.411919 27144 tablet_copy_client.cc:523] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:33.412755 27144 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:33.430678 27144 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:33.430881 27144 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:33.431023 27144 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20240501 14:10:33.431362 27144 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:33.431463 27144 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:33.431551 27144 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:33.431843 27144 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.004s
I20240501 14:10:33.432268 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-ba3621a3f5df43eca96826da75fc31d0 received from {username='slave'} at 127.6.153.202:44465
I20240501 14:10:33.432333 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session c958143102d04279ac25aa56214d0d4b-ba3621a3f5df43eca96826da75fc31d0 on tablet ba3621a3f5df43eca96826da75fc31d0 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:33.439268 27135 ts_tablet_manager.cc:902] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:33.439637 27135 tablet_copy_client.cc:308] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:33.442363 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet d25ef07189ac43749efc549299dc89f8 from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:56835)
I20240501 14:10:33.442448 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet d25ef07189ac43749efc549299dc89f8 from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:56835: session id = a308c0ed71f44f57967c97239d1198f7-d25ef07189ac43749efc549299dc89f8
I20240501 14:10:33.443056 23752 tablet_copy_source_session.cc:205] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:33.443969 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:33.445513 27135 tablet_copy_client.cc:783] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:33.445634 27135 tablet_copy_client.cc:647] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:33.446570 27135 tablet_copy_client.cc:523] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:33.447454 27135 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:33.466884 27135 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:33.467120 27135 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:33.467267 27135 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.000s
I20240501 14:10:33.467654 27135 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:33.467783 27135 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:33.467885 27135 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:33.468114 27135 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:10:33.468525 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-d25ef07189ac43749efc549299dc89f8 received from {username='slave'} at 127.6.153.201:56835
I20240501 14:10:33.468605 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-d25ef07189ac43749efc549299dc89f8 on tablet d25ef07189ac43749efc549299dc89f8 with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:33.514086 27135 ts_tablet_manager.cc:902] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:33.514508 27135 tablet_copy_client.cc:308] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:46669
I20240501 14:10:33.519486 24148 tablet_copy_service.cc:140] P 166d7170d98944e88de2eccda9d30545: Received BeginTabletCopySession request for tablet de25122a01e845d2b35e4a1023789d3e from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:41233)
I20240501 14:10:33.519584 24148 tablet_copy_service.cc:161] P 166d7170d98944e88de2eccda9d30545: Beginning new tablet copy session on tablet de25122a01e845d2b35e4a1023789d3e from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:41233: session id = a308c0ed71f44f57967c97239d1198f7-de25122a01e845d2b35e4a1023789d3e
I20240501 14:10:33.521272 24148 tablet_copy_source_session.cc:205] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:33.522207 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:33.524497 27135 tablet_copy_client.cc:783] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:33.524650 27135 tablet_copy_client.cc:647] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:33.526015 27135 tablet_copy_client.cc:523] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:33.527168 27135 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:33.552016 27144 ts_tablet_manager.cc:902] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:33.552407 27144 tablet_copy_client.cc:308] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:38101
I20240501 14:10:33.556501 27135 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:33.556738 27135 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:33.556895 27135 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.030s user 0.019s sys 0.001s
I20240501 14:10:33.557304 27135 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:33.557435 27135 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:33.557531 27135 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:33.563032 27135 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240501 14:10:33.564710 24148 tablet_copy_service.cc:342] P 166d7170d98944e88de2eccda9d30545: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-de25122a01e845d2b35e4a1023789d3e received from {username='slave'} at 127.6.153.201:41233
I20240501 14:10:33.564800 24148 tablet_copy_service.cc:433] P 166d7170d98944e88de2eccda9d30545: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-de25122a01e845d2b35e4a1023789d3e on tablet de25122a01e845d2b35e4a1023789d3e with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:33.565969 24280 tablet_copy_service.cc:140] P dbeb6a07fa7a44019ae00c169c414529: Received BeginTabletCopySession request for tablet 34a278486c8b447d8dc4c3241743de60 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:45777)
I20240501 14:10:33.566062 24280 tablet_copy_service.cc:161] P dbeb6a07fa7a44019ae00c169c414529: Beginning new tablet copy session on tablet 34a278486c8b447d8dc4c3241743de60 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:45777: session id = c958143102d04279ac25aa56214d0d4b-34a278486c8b447d8dc4c3241743de60
I20240501 14:10:33.566927 24280 tablet_copy_source_session.cc:205] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:33.567986 27135 ts_tablet_manager.cc:902] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:33.568327 27135 tablet_copy_client.cc:308] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:44313
I20240501 14:10:33.568755 24016 tablet_copy_service.cc:140] P 8ebe0a098b0f4ed4861a5605717fecbd: Received BeginTabletCopySession request for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:51351)
I20240501 14:10:33.568864 24016 tablet_copy_service.cc:161] P 8ebe0a098b0f4ed4861a5605717fecbd: Beginning new tablet copy session on tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:51351: session id = a308c0ed71f44f57967c97239d1198f7-0fc754b4d0f144c4ba0bd9a1b77f7bb4
I20240501 14:10:33.569461 27144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:33.569674 24016 tablet_copy_source_session.cc:205] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:33.570660 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:33.571630 27144 tablet_copy_client.cc:783] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:33.571758 27144 tablet_copy_client.cc:647] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:33.572708 27135 tablet_copy_client.cc:783] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:33.572854 27135 tablet_copy_client.cc:647] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:33.578697 27144 tablet_copy_client.cc:523] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:33.579870 27144 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:33.580034 27135 tablet_copy_client.cc:523] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:33.581080 27135 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:33.601683 27144 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:33.601998 27144 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:33.602159 27144 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.000s
I20240501 14:10:33.602525 27144 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:33.602631 27144 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:33.602723 27144 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:33.603055 27144 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:33.603590 24280 tablet_copy_service.cc:342] P dbeb6a07fa7a44019ae00c169c414529: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-34a278486c8b447d8dc4c3241743de60 received from {username='slave'} at 127.6.153.202:45777
I20240501 14:10:33.603660 24280 tablet_copy_service.cc:433] P dbeb6a07fa7a44019ae00c169c414529: ending tablet copy session c958143102d04279ac25aa56214d0d4b-34a278486c8b447d8dc4c3241743de60 on tablet 34a278486c8b447d8dc4c3241743de60 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:33.608204 27135 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:33.608451 27135 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:33.608615 27135 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.028s user 0.021s sys 0.003s
I20240501 14:10:33.609019 27135 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:33.609164 27135 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:33.609270 27135 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:33.609555 27135 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:33.610100 24016 tablet_copy_service.cc:342] P 8ebe0a098b0f4ed4861a5605717fecbd: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-0fc754b4d0f144c4ba0bd9a1b77f7bb4 received from {username='slave'} at 127.6.153.201:51351
I20240501 14:10:33.610186 24016 tablet_copy_service.cc:433] P 8ebe0a098b0f4ed4861a5605717fecbd: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-0fc754b4d0f144c4ba0bd9a1b77f7bb4 on tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:33.869230 25808 raft_consensus.cc:1203] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:33.896076 25941 raft_consensus.cc:1203] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:33.944701 25808 raft_consensus.cc:1203] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:33.974637 25808 raft_consensus.cc:1203] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:33.991955 25808 raft_consensus.cc:1203] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:34.080830 25940 raft_consensus.cc:1203] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:34.343214 27236 raft_consensus.cc:1050] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:34.343686 27236 consensus_queue.cc:237] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:34.345374 24258 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.345358 24127 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.345567 23730 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.345562 23862 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.345849 27304 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.346063 27236 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.346148 25808 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.346374 27305 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.346589 27236 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.347182 27235 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.347913 24392 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.348026 24524 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.348409 27236 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.348803 27236 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.350463 27305 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.350636 24127 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.351001 23862 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.351099 24392 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.353662 25808 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.354020 23581 catalog_manager.cc:5484] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.354322 24258 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.355890 23730 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.357218 24524 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.362340 23994 consensus_queue.cc:237] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:34.363700 23862 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.363803 25808 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.364059 27306 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.364117 24127 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.364254 27236 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.364362 24392 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.364524 23730 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.364753 27306 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.365514 27306 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.365634 27236 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.365743 24524 raft_consensus.cc:1261] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.366101 27306 consensus_queue.cc:1035] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.367502 27306 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.367882 24127 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.368860 23730 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.369735 23862 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.370744 23586 catalog_manager.cc:5484] T bb091eaf92de4f9bbf110f489a9f21bd P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.370889 23577 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bb091eaf92de4f9bbf110f489a9f21bd with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:34.372635 25808 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.372637 24524 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.373180 24392 raft_consensus.cc:2932] T bb091eaf92de4f9bbf110f489a9f21bd P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.375830 24239 tablet_service.cc:1514] Processing DeleteTablet for tablet bb091eaf92de4f9bbf110f489a9f21bd with delete_type TABLET_DATA_TOMBSTONED (TS dbeb6a07fa7a44019ae00c169c414529 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:51508
I20240501 14:10:34.376173 27318 tablet_replica.cc:330] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: stopping tablet replica
I20240501 14:10:34.376341 27318 raft_consensus.cc:2229] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:34.376575 27318 raft_consensus.cc:2258] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:34.377174 27318 ts_tablet_manager.cc:1875] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:34.378942 27318 ts_tablet_manager.cc:1888] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:34.379000 27318 log.cc:1196] T bb091eaf92de4f9bbf110f489a9f21bd P dbeb6a07fa7a44019ae00c169c414529: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/bb091eaf92de4f9bbf110f489a9f21bd
I20240501 14:10:34.379407 23575 catalog_manager.cc:4830] TS dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): tablet bb091eaf92de4f9bbf110f489a9f21bd (table rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]) successfully deleted
I20240501 14:10:34.394835 27235 raft_consensus.cc:1050] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:34.395234 27235 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:34.396849 25808 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.397087 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.397285 24127 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.397251 26820 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.397388 24258 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.397670 26820 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.397818 27305 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.397949 27304 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.398568 24524 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.399634 27306 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.399960 23862 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.400084 23730 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.400377 27305 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.400548 27236 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.401405 27305 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.401701 24258 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.403175 25808 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.403293 24392 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.403707 24127 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.405206 23862 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.405419 23581 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:34.407434 24524 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.410419 23730 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.413619 23994 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:34.413847 27018 raft_consensus.cc:1050] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:34.414309 27018 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:34.415221 24524 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.415251 25808 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.415277 24127 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.415640 27304 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.415794 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.416087 27306 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.416364 27304 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.417098 23730 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.417443 27305 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.417822 23862 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.418192 27305 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.418267 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.418193 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.419104 23864 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.418959 27304 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.419152 27157 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.419411 27258 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.420500 23577 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:34.420500 23731 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.420889 27157 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.421209 25808 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.422616 24392 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.422705 23586 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:34.423300 27236 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.424746 23862 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.425243 24524 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.427067 24239 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with delete_type TABLET_DATA_TOMBSTONED (TS dbeb6a07fa7a44019ae00c169c414529 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:51508
I20240501 14:10:34.427274 27318 tablet_replica.cc:330] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: stopping tablet replica
I20240501 14:10:34.427500 27318 raft_consensus.cc:2229] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:34.427742 27318 raft_consensus.cc:2258] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:34.428771 27318 ts_tablet_manager.cc:1875] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:34.428978 24524 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.429311 27328 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.429962 24127 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.429347 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.430821 27318 ts_tablet_manager.cc:1888] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:34.430924 27318 log.cc:1196] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P dbeb6a07fa7a44019ae00c169c414529: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/0fc754b4d0f144c4ba0bd9a1b77f7bb4
I20240501 14:10:34.431036 27328 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.431802 23575 catalog_manager.cc:4830] TS dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (table rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]) successfully deleted
I20240501 14:10:34.431912 24258 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.432269 27329 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.433534 25808 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.433836 27329 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.434741 24258 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.435815 27328 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.436564 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.436779 23994 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.437760 23731 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.438495 23590 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.439114 24524 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.442229 23864 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.443061 25808 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.447036 24127 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:34.448581 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.448697 24524 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.448966 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.448966 27332 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.449239 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.449419 27258 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.449610 27157 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.449717 23864 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.449718 27258 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.450038 27258 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.451889 25808 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.451870 27331 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.452170 23731 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.452288 27332 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.452988 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.453313 25808 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.454244 23994 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.454684 23586 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.454903 24524 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.456912 23864 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.458492 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet de25122a01e845d2b35e4a1023789d3e with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:34.458962 24239 tablet_service.cc:1514] Processing DeleteTablet for tablet de25122a01e845d2b35e4a1023789d3e with delete_type TABLET_DATA_TOMBSTONED (TS dbeb6a07fa7a44019ae00c169c414529 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:51508
I20240501 14:10:34.459174 27318 tablet_replica.cc:330] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: stopping tablet replica
I20240501 14:10:34.459295 27318 raft_consensus.cc:2229] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:34.459511 27318 raft_consensus.cc:2258] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:34.460731 27318 ts_tablet_manager.cc:1875] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:34.462268 27318 ts_tablet_manager.cc:1888] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:34.462329 27318 log.cc:1196] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-4/wal/wals/de25122a01e845d2b35e4a1023789d3e
I20240501 14:10:34.462716 23575 catalog_manager.cc:4830] TS dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): tablet de25122a01e845d2b35e4a1023789d3e (table rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]) successfully deleted
I20240501 14:10:34.468196 27198 raft_consensus.cc:1050] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:34.468734 27198 consensus_queue.cc:237] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:34.470530 24258 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.470575 24127 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.470644 24392 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.470904 27198 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.471199 23864 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.471199 26706 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.471765 26706 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.471879 27198 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.472198 25940 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.473138 24524 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.473515 27237 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.473647 23994 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.473739 26706 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.474082 26706 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.476240 26706 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.477020 23994 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.478696 24258 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.478837 24524 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.479207 25940 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.480459 24126 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.480784 24392 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.481271 23587 catalog_manager.cc:5484] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.483031 23864 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.488902 23731 consensus_queue.cc:237] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:34.490593 25940 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.490690 23994 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.490867 24392 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.490947 27356 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.491250 24258 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.491289 24524 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.491463 27237 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.491760 27237 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.492360 27358 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.493122 27270 raft_consensus.cc:1050] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:34.493639 27270 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:34.493919 27356 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.495011 23864 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.495586 27357 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.495757 25940 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.496091 27269 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.496308 26706 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.496901 24521 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.497295 24392 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.498086 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ba3621a3f5df43eca96826da75fc31d0 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:34.498180 25941 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.498787 24258 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.499508 23586 catalog_manager.cc:5484] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.500229 23862 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.501036 23863 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.501540 27065 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.501628 23995 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.501974 24392 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.502095 23994 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.502775 24524 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.502763 27065 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.503254 24126 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.503479 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet ba3621a3f5df43eca96826da75fc31d0 with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:34.503685 27269 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.503861 27380 tablet_replica.cc:330] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:34.503998 27380 raft_consensus.cc:2229] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:34.503996 27269 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.504580 27065 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.505841 27380 raft_consensus.cc:2258] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:34.506074 23731 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.506680 27380 ts_tablet_manager.cc:1875] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:34.506664 27270 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.509517 27380 ts_tablet_manager.cc:1888] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:34.510385 27269 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.511672 24392 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.512143 23730 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.512770 23995 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.512691 23864 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.514374 23584 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.515260 25940 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.511209 24126 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.519477 27380 log.cc:1196] T ba3621a3f5df43eca96826da75fc31d0 P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/ba3621a3f5df43eca96826da75fc31d0
I20240501 14:10:34.520844 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet ba3621a3f5df43eca96826da75fc31d0 (table rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]) successfully deleted
I20240501 14:10:34.520954 24524 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.524533 24258 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:34.525875 25940 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.525985 24392 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.525995 23730 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.526338 27065 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.526434 23995 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.526536 27377 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.526690 23864 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.526764 27269 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.527038 27378 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.527211 27379 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.527390 24524 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:34.528316 27378 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:34.529407 27269 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.530467 23730 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.530575 24524 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.530838 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 34a278486c8b447d8dc4c3241743de60 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:34.531517 24392 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.534242 23590 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.535482 23864 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.535681 25940 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.536530 23995 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:34.537802 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a278486c8b447d8dc4c3241743de60 with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:34.538033 27380 tablet_replica.cc:330] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:34.538281 27380 raft_consensus.cc:2229] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:34.538524 27380 raft_consensus.cc:2258] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:34.540025 27380 ts_tablet_manager.cc:1875] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:34.541991 27380 ts_tablet_manager.cc:1888] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:34.542047 27380 log.cc:1196] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/34a278486c8b447d8dc4c3241743de60
I20240501 14:10:34.542461 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet 34a278486c8b447d8dc4c3241743de60 (table rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]) successfully deleted
I20240501 14:10:34.584331 26706 raft_consensus.cc:1050] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:34.584816 26706 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:34.586350 24524 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.586505 23995 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.586614 24392 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.586630 25808 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.586884 27358 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.587028 23864 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.587364 27358 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.587627 27198 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.587774 27237 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.588052 24126 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.588407 27358 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.588541 27198 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.590559 24258 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:34.590555 27198 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.590917 27357 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:34.590950 24126 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.591744 24392 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.592113 24258 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.593158 23864 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.593783 23995 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.595078 23589 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.598011 24524 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.599536 25808 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:34.956126 23864 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:34.957918 23995 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.958154 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.958272 23730 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.958272 24521 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.958657 24392 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.958622 27425 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.958876 24126 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.959121 26894 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.959409 27426 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.959532 27427 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.959684 27425 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.960379 27424 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.962333 27424 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:34.962720 23730 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:34.963052 24126 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:34.964637 24392 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:34.965399 23586 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:34.965454 23995 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:34.968518 24521 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:34.971004 24258 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:34.971567 23864 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:34.973749 24521 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:34.973846 23995 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:34.974232 24392 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:34.974619 27429 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:34.974726 23730 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:34.974932 27429 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:34.975297 24126 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:34.975695 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:34.977587 27424 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:34.977864 27429 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:34.978199 27429 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 14:10:34.979570 23797 consensus_peers.cc:485] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21c8e4484fc94645869c7a912df55e4c. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:34.980702 27425 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:34.981398 23730 consensus_queue.cc:237] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:34.981549 27426 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:34.983462 24392 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:34.983836 24390 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.983880 24521 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.983980 23994 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:34.984220 27237 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.984273 24126 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.984829 24258 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.985437 27357 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.985742 27237 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.986205 27237 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.986503 23577 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 21c8e4484fc94645869c7a912df55e4c with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:34.986930 23995 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.987797 27358 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.987815 24521 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:34.987896 24260 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:34.988209 23588 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:34.989317 23864 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:34.990200 27237 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:34.990296 23731 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:34.992233 26706 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:34.993947 24258 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:34.994544 24521 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:34.994761 23864 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:34.995774 24128 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:34.998167 23588 catalog_manager.cc:5484] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } }
I20240501 14:10:35.002158 23731 consensus_queue.cc:237] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:35.002938 24392 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
W20240501 14:10:35.004446 23665 consensus_peers.cc:485] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37fda65c1f5240ce9f886d57921a03f0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:35.004635 23864 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.004751 24258 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.004776 24521 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.005120 27198 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.005223 27357 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.005704 27237 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.007515 24392 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.007846 27198 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.008601 26706 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.009683 24128 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.010264 23575 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 37fda65c1f5240ce9f886d57921a03f0 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:35.010617 27358 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:35.010558 24390 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.010737 23994 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.011183 24258 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.013255 24521 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.013760 23581 catalog_manager.cc:5484] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.013702 23864 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.015836 24128 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.016566 27357 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:35.017235 24128 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.018854 23994 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.020184 23994 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.069326 23731 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:35.070966 24521 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:35.071238 23994 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:35.071527 27455 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:35.071729 24392 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:35.071806 27356 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:35.072263 25808 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:35.072261 23864 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:35.072732 24258 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:35.072965 27356 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:35.073128 27455 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:35.073583 27357 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:35.073766 26706 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:35.075469 26706 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:35.076202 23864 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:35.077853 24392 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:35.077950 24521 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:35.078063 24258 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:35.080336 23586 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:35.081066 25808 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:35.082713 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d25ef07189ac43749efc549299dc89f8 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:35.083174 23994 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:35.083987 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet d25ef07189ac43749efc549299dc89f8 with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:35.090204 27467 tablet_replica.cc:330] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:35.090324 27467 raft_consensus.cc:2229] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:35.090494 27467 raft_consensus.cc:2258] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:35.091078 27467 ts_tablet_manager.cc:1875] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:35.093048 27467 ts_tablet_manager.cc:1888] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:35.093108 27467 log.cc:1196] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/d25ef07189ac43749efc549299dc89f8
I20240501 14:10:35.093457 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet d25ef07189ac43749efc549299dc89f8 (table rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]) successfully deleted
I20240501 14:10:35.413949 27483 ts_tablet_manager.cc:902] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:35.414441 27483 tablet_copy_client.cc:308] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:35.414932 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet 37fda65c1f5240ce9f886d57921a03f0 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:44465)
I20240501 14:10:35.415058 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet 37fda65c1f5240ce9f886d57921a03f0 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:44465: session id = c958143102d04279ac25aa56214d0d4b-37fda65c1f5240ce9f886d57921a03f0
I20240501 14:10:35.415930 23752 tablet_copy_source_session.cc:205] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:35.416790 27483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:35.418524 27483 tablet_copy_client.cc:783] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:35.418669 27483 tablet_copy_client.cc:647] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:35.419677 27483 tablet_copy_client.cc:523] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:35.420593 27483 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:35.438578 27483 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:35.438834 27483 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:35.439019 27483 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.005s
I20240501 14:10:35.439484 27483 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:35.439621 27483 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:35.439708 27483 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:35.440063 27483 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240501 14:10:35.440529 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-37fda65c1f5240ce9f886d57921a03f0 received from {username='slave'} at 127.6.153.202:44465
I20240501 14:10:35.440609 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session c958143102d04279ac25aa56214d0d4b-37fda65c1f5240ce9f886d57921a03f0 on tablet 37fda65c1f5240ce9f886d57921a03f0 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:35.452467 24392 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } }
I20240501 14:10:35.454167 23731 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:35.454167 24258 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:35.454187 23994 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:35.454210 24128 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:35.454504 24521 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:35.454533 27497 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:35.454835 23864 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:35.454823 27499 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:35.455021 27497 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:35.455343 27301 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:35.455502 27302 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:35.455612 27497 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:35.458048 27497 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.458413 23731 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.459831 23864 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.460520 23994 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.461269 24128 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.462214 24258 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.464094 23589 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.466598 24521 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } }
I20240501 14:10:35.469743 24392 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
W20240501 14:10:35.471297 24325 consensus_peers.cc:485] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76fbefe89b8344e1a09101d404f32d9e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:35.471792 23731 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.472476 24258 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.472594 27499 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.472693 23864 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.472739 23994 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.472855 27497 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.473426 27497 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.473587 27506 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.474503 24521 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.474498 24128 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:35.475466 27506 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.475634 27499 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:35.476402 27498 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.477612 24521 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.478194 24128 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.478461 23574 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 76fbefe89b8344e1a09101d404f32d9e with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:35.481442 24258 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.481537 23731 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.481863 23588 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.485096 27518 ts_tablet_manager.cc:902] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427)
I20240501 14:10:35.485343 23864 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.485756 27518 tablet_copy_client.cc:308] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:34427
I20240501 14:10:35.487444 23994 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:35.492758 23884 tablet_copy_service.cc:140] P 2502b9e8098b4bc5af7204835fce444b: Received BeginTabletCopySession request for tablet 21c8e4484fc94645869c7a912df55e4c from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:48385)
I20240501 14:10:35.492856 23884 tablet_copy_service.cc:161] P 2502b9e8098b4bc5af7204835fce444b: Beginning new tablet copy session on tablet 21c8e4484fc94645869c7a912df55e4c from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:48385: session id = 6d832f3f723641778e60b3fd855ff98e-21c8e4484fc94645869c7a912df55e4c
I20240501 14:10:35.493556 23884 tablet_copy_source_session.cc:205] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:35.494531 27518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:35.497025 27518 tablet_copy_client.cc:783] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:35.497228 27518 tablet_copy_client.cc:647] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:35.498196 27518 tablet_copy_client.cc:523] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:35.499267 27518 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:35.517261 27518 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:35.517481 27518 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:35.517627 27518 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.000s
I20240501 14:10:35.517963 27518 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:35.518065 27518 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:35.518141 27518 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:35.518414 27518 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 14:10:35.518824 23884 tablet_copy_service.cc:342] P 2502b9e8098b4bc5af7204835fce444b: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-21c8e4484fc94645869c7a912df55e4c received from {username='slave'} at 127.6.153.200:48385
I20240501 14:10:35.518909 23884 tablet_copy_service.cc:433] P 2502b9e8098b4bc5af7204835fce444b: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-21c8e4484fc94645869c7a912df55e4c on tablet 21c8e4484fc94645869c7a912df55e4c with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:35.855954 25940 raft_consensus.cc:1203] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:35.866493 26072 raft_consensus.cc:1203] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:35.898552 27483 ts_tablet_manager.cc:902] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:35.898994 27483 tablet_copy_client.cc:308] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:35.899428 24412 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet 76fbefe89b8344e1a09101d404f32d9e from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:59269)
I20240501 14:10:35.899544 24412 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet 76fbefe89b8344e1a09101d404f32d9e from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:59269: session id = c958143102d04279ac25aa56214d0d4b-76fbefe89b8344e1a09101d404f32d9e
I20240501 14:10:35.900429 24412 tablet_copy_source_session.cc:205] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:35.901278 27483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:35.903406 27483 tablet_copy_client.cc:783] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:35.903631 27483 tablet_copy_client.cc:647] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:35.904978 27483 tablet_copy_client.cc:523] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:35.906087 27483 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:35.928005 27483 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:35.928238 27483 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:35.928400 27483 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.022s user 0.017s sys 0.004s
I20240501 14:10:35.928741 27483 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:35.928850 27483 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:35.928942 27483 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:35.929216 27483 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:35.929615 24412 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-76fbefe89b8344e1a09101d404f32d9e received from {username='slave'} at 127.6.153.202:59269
I20240501 14:10:35.929692 24412 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session c958143102d04279ac25aa56214d0d4b-76fbefe89b8344e1a09101d404f32d9e on tablet 76fbefe89b8344e1a09101d404f32d9e with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:36.263239 27547 raft_consensus.cc:1050] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:36.263718 27547 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:36.265139 26072 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.265192 23994 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.265650 27548 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.266098 23731 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.266160 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.266218 24392 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.266500 24128 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.266655 24524 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.266942 27586 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.267131 27587 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.267244 27588 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.267361 27547 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.267477 26894 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.267962 27587 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.270107 27547 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.270787 23994 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.271616 24258 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.272629 26072 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.272656 24128 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.273386 23584 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:36.273658 23731 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.274845 24524 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.275346 24392 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.279839 23864 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:36.281457 23994 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.281524 24392 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.281832 27588 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.281924 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.282116 24524 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.282276 23731 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.282294 27586 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.282547 27588 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.282836 27588 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.283242 27588 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.284438 26072 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.284796 27547 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.285048 23731 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.286326 23577 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 21c8e4484fc94645869c7a912df55e4c with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:36.284787 27586 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20240501 14:10:36.287370 24392 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.287554 24524 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.287624 23588 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.287778 23994 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.291283 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 21c8e4484fc94645869c7a912df55e4c with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:36.292107 26072 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.292930 24258 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:36.295534 27601 tablet_replica.cc:330] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:36.295703 27601 raft_consensus.cc:2229] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:36.296273 27601 raft_consensus.cc:2258] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:36.296912 27601 ts_tablet_manager.cc:1875] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:36.298404 27601 ts_tablet_manager.cc:1888] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:36.298465 27601 log.cc:1196] T 21c8e4484fc94645869c7a912df55e4c P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/21c8e4484fc94645869c7a912df55e4c
I20240501 14:10:36.298866 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet 21c8e4484fc94645869c7a912df55e4c (table rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]) successfully deleted
I20240501 14:10:36.361275 27198 raft_consensus.cc:1050] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:36.361831 27198 consensus_queue.cc:237] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:36.363497 25940 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.363898 23994 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.364558 24392 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.364569 24128 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.364285 27615 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.364959 27617 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.365033 24524 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.365149 27618 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.365497 27616 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.365660 24258 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.365660 27618 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.366115 27618 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.367517 23864 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.369472 27618 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.369860 27614 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.370049 23994 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.370638 24258 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.371908 24392 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.372109 23864 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.372467 25940 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.374547 23587 catalog_manager.cc:5484] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.375243 24524 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.375475 24128 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.415410 25940 raft_consensus.cc:1203] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:36.589402 24258 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:36.590948 23731 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:36.591080 24392 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:36.591379 23994 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:36.591481 27641 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:36.591648 27642 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:36.591681 24524 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:36.592082 27642 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:36.592218 27377 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:36.592495 23864 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:36.592825 24128 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:36.593602 27642 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:36.593736 27377 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:36.595439 27643 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:36.596185 23864 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:36.596758 24128 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:36.598093 23994 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:36.598223 23731 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:36.600019 23584 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:36.602488 24392 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:36.602984 24524 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:36.606686 24258 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
W20240501 14:10:36.609097 24193 consensus_peers.cc:485] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3377f84deb6249668991e5a3f6d63876. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:36.609186 23864 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:36.609186 23994 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:36.609568 24128 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:36.609898 27655 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:36.609937 24524 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:36.610980 27655 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:36.611311 27640 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:36.611493 27655 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:36.612429 24392 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:36.612761 27641 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:36.613240 23731 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:36.613595 27655 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:36.614125 27640 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:36.614521 23864 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:36.615949 23575 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 3377f84deb6249668991e5a3f6d63876 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:36.616220 24128 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:36.616420 23994 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:36.617089 24392 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:36.617148 23582 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:36.618692 24524 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:36.623503 23731 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:36.835211 23731 consensus_queue.cc:237] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:36.837216 24524 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.837445 25940 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.837491 23864 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.837698 27615 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.837801 24392 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.837862 27618 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.837905 23994 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.838836 27615 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.838879 24258 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.839075 27616 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.839398 27616 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.840173 27615 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.840579 27615 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.840919 23864 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.843602 24524 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.843866 23586 catalog_manager.cc:5484] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:36.844712 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 37fda65c1f5240ce9f886d57921a03f0 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:36.845155 24258 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.845352 23994 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.850065 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:36.850704 27677 tablet_replica.cc:330] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:36.850664 24392 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.850838 27677 raft_consensus.cc:2229] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:36.851024 27677 raft_consensus.cc:2258] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:36.851658 27677 ts_tablet_manager.cc:1875] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:36.852922 25940 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.854090 27677 ts_tablet_manager.cc:1888] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:36.854149 27677 log.cc:1196] T 37fda65c1f5240ce9f886d57921a03f0 P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/37fda65c1f5240ce9f886d57921a03f0
I20240501 14:10:36.856149 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet 37fda65c1f5240ce9f886d57921a03f0 (table rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]) successfully deleted
I20240501 14:10:36.915313 27631 raft_consensus.cc:1050] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:36.915786 27631 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:36.917856 25940 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.917923 23994 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.918040 24258 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.918073 24128 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.918243 27687 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.918437 27517 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.918579 23731 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.919054 24524 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.919035 27517 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.919580 27687 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.919852 23864 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:36.920064 27631 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.920275 27517 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.920905 27686 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:36.922623 27686 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.923421 24524 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.923493 24128 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.924808 25940 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.924942 23994 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.925674 23731 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.926657 24258 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.927119 23582 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.930488 23864 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.932688 24390 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:36.934073 25940 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.934417 23994 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.934464 27631 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.934572 24524 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.934826 27631 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.935076 24258 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.935163 27631 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.935469 27631 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.935883 23731 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.936365 27687 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.936666 23864 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:36.936730 27687 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.937017 27517 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:36.937227 23994 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.938139 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 76fbefe89b8344e1a09101d404f32d9e with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:36.938458 24258 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.939473 23864 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.939940 24524 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.940191 23586 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.941597 25940 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.942909 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 76fbefe89b8344e1a09101d404f32d9e with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:36.943125 27677 tablet_replica.cc:330] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:36.943243 27677 raft_consensus.cc:2229] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:36.943432 27677 raft_consensus.cc:2258] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:36.943262 23731 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:36.944519 27677 ts_tablet_manager.cc:1875] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:36.946251 27677 ts_tablet_manager.cc:1888] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:36.946298 27677 log.cc:1196] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/76fbefe89b8344e1a09101d404f32d9e
I20240501 14:10:36.946660 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet 76fbefe89b8344e1a09101d404f32d9e (table rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]) successfully deleted
I20240501 14:10:37.005275 27695 ts_tablet_manager.cc:902] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:37.005733 27695 tablet_copy_client.cc:308] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:38101
I20240501 14:10:37.006188 24280 tablet_copy_service.cc:140] P dbeb6a07fa7a44019ae00c169c414529: Received BeginTabletCopySession request for tablet 3377f84deb6249668991e5a3f6d63876 from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:33507)
I20240501 14:10:37.006282 24280 tablet_copy_service.cc:161] P dbeb6a07fa7a44019ae00c169c414529: Beginning new tablet copy session on tablet 3377f84deb6249668991e5a3f6d63876 from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:33507: session id = a308c0ed71f44f57967c97239d1198f7-3377f84deb6249668991e5a3f6d63876
I20240501 14:10:37.006964 24280 tablet_copy_source_session.cc:205] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:37.007772 27695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:37.009373 27695 tablet_copy_client.cc:783] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:37.009510 27695 tablet_copy_client.cc:647] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:37.010452 27695 tablet_copy_client.cc:523] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:37.011322 27695 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:37.028056 27695 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:37.028276 27695 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:37.028420 27695 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.017s user 0.017s sys 0.000s
I20240501 14:10:37.028739 27695 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:37.028841 27695 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:37.028941 27695 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:37.029261 27695 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:10:37.029693 24280 tablet_copy_service.cc:342] P dbeb6a07fa7a44019ae00c169c414529: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-3377f84deb6249668991e5a3f6d63876 received from {username='slave'} at 127.6.153.201:33507
I20240501 14:10:37.029771 24280 tablet_copy_service.cc:433] P dbeb6a07fa7a44019ae00c169c414529: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-3377f84deb6249668991e5a3f6d63876 on tablet 3377f84deb6249668991e5a3f6d63876 with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:37.067510 24390 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } }
I20240501 14:10:37.069162 23864 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:37.069180 23731 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:37.069172 23994 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:37.069610 27685 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:37.069919 27685 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:37.070153 27685 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:37.071884 24524 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:37.071924 24128 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:37.071916 24258 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 14:10:37.072405 27517 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:37.072594 27631 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:37.072724 27686 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:37.074859 27684 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:37.075443 23731 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:37.075431 24258 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:37.077243 23864 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:37.077270 23994 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:37.079262 23588 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:37.084209 24390 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:37.085069 24128 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:37.087069 24128 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:37.087128 23731 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:37.087149 24258 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 14:10:37.087455 24325 consensus_peers.cc:485] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 84da4677d89b42fe8c2418cd156c75ae. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:37.087559 27685 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:37.089102 27685 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:37.089243 24524 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:37.089680 23994 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:37.090368 27684 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:37.091096 27685 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 14:10:37.091352 23864 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 14:10:37.091377 27685 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:37.091714 27685 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 14:10:37.092553 24524 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } }
I20240501 14:10:37.093726 27631 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:37.094082 24128 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:37.095824 24258 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:37.096590 23731 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:37.098039 23589 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:37.098934 24524 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:37.099475 23574 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 84da4677d89b42fe8c2418cd156c75ae with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:37.099853 23994 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:37.100570 23862 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:37.393409 25808 raft_consensus.cc:1203] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:37.616395 27753 ts_tablet_manager.cc:902] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:37.617007 27753 tablet_copy_client.cc:308] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:37.617451 24412 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet 84da4677d89b42fe8c2418cd156c75ae from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:41349)
I20240501 14:10:37.617545 24412 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet 84da4677d89b42fe8c2418cd156c75ae from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:41349: session id = 6d832f3f723641778e60b3fd855ff98e-84da4677d89b42fe8c2418cd156c75ae
I20240501 14:10:37.618254 24412 tablet_copy_source_session.cc:205] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:37.619032 27753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:37.620784 27753 tablet_copy_client.cc:783] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:37.620929 27753 tablet_copy_client.cc:647] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:37.621915 27753 tablet_copy_client.cc:523] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:37.622862 27753 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:37.640240 27753 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:37.640439 27753 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:37.640564 27753 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.001s
I20240501 14:10:37.640906 27753 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:37.641031 27753 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:37.641135 27753 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:37.641521 27753 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:10:37.641930 24412 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-84da4677d89b42fe8c2418cd156c75ae received from {username='slave'} at 127.6.153.200:41349
I20240501 14:10:37.641999 24412 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-84da4677d89b42fe8c2418cd156c75ae on tablet 84da4677d89b42fe8c2418cd156c75ae with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:38.000335 27741 raft_consensus.cc:1050] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:38.000824 27741 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:38.002316 25808 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.002394 23731 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.002754 23864 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.002847 24128 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.003171 24524 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.003186 27780 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.003405 27783 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.003583 27786 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.003695 23994 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.003713 24390 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.003865 27786 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.004024 27785 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.004163 27741 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.004310 27784 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.006968 27741 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.007279 24524 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.008013 23731 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.008096 23864 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.009542 23994 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.009982 24128 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.010785 23582 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.010972 24390 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.014715 25808 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.019243 24258 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:38.020651 23731 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.020784 23994 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.020911 23864 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.021070 24390 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.021256 27786 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.021404 27780 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.021535 27783 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.021701 24524 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.021701 25808 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.021939 27782 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.022120 27786 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.023782 27786 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.024272 27785 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.024945 23994 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.025070 23731 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.025238 24524 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.026778 23864 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.026984 24390 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.029120 23589 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.029567 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3377f84deb6249668991e5a3f6d63876 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:38.030773 25808 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:38.032058 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 3377f84deb6249668991e5a3f6d63876 with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:38.032377 27799 tablet_replica.cc:330] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:38.032501 27799 raft_consensus.cc:2229] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:38.032675 27799 raft_consensus.cc:2258] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:38.033324 27799 ts_tablet_manager.cc:1875] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:38.035521 27799 ts_tablet_manager.cc:1888] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:38.035599 27799 log.cc:1196] T 3377f84deb6249668991e5a3f6d63876 P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/3377f84deb6249668991e5a3f6d63876
I20240501 14:10:38.036095 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet 3377f84deb6249668991e5a3f6d63876 (table rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]) successfully deleted
I20240501 14:10:38.082412 26071 raft_consensus.cc:1203] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 14:10:38.618342 27800 raft_consensus.cc:1050] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:38.618806 27800 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:38.620395 26071 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.620584 23994 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.620795 23864 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.620867 24524 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.621093 27823 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.621476 27825 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.621563 23731 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.621615 24258 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.621774 27800 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.621948 27825 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.622282 27822 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.622612 27719 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.623919 24128 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 14:10:38.624384 27823 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 14:10:38.625670 27823 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.626313 26071 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.627027 24258 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.627432 23994 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.629272 23864 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.630095 23586 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.630539 24524 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.631484 24128 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.631774 23731 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.638206 24392 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:38.639643 23731 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.639757 26071 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.639832 23864 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.640103 27824 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.640390 24258 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.640405 24524 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.640412 27719 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.640980 27719 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.641098 27824 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.641168 23994 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 14:10:38.641373 27822 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.641819 27822 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 14:10:38.643239 27719 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.643772 23864 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.644502 23731 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.645081 26071 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.645321 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 84da4677d89b42fe8c2418cd156c75ae with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:38.646621 24524 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.646669 24258 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.647818 23581 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.649874 23994 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:38.651186 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 84da4677d89b42fe8c2418cd156c75ae with delete_type TABLET_DATA_TOMBSTONED (TS 166d7170d98944e88de2eccda9d30545 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:38476
I20240501 14:10:38.651569 27838 tablet_replica.cc:330] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: stopping tablet replica
I20240501 14:10:38.651737 27838 raft_consensus.cc:2229] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:38.652000 27838 raft_consensus.cc:2258] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:38.652717 27838 ts_tablet_manager.cc:1875] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:38.654798 27838 ts_tablet_manager.cc:1888] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 14:10:38.654871 27838 log.cc:1196] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-3/wal/wals/84da4677d89b42fe8c2418cd156c75ae
I20240501 14:10:38.655299 23576 catalog_manager.cc:4830] TS 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): tablet 84da4677d89b42fe8c2418cd156c75ae (table rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]) successfully deleted
I20240501 14:10:39.057209 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:39.057407 24504 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:39.061091 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:39.061602 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:39.062198 23844 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:39.078905 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:39.079064 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:39.088114 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:39.109972 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:39.121896 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:39.213722 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:39.215255 23844 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:39.216794 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:39.222146 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:39.224574 24504 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:39.248085 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:39.248140 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:39.251346 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:39.251962 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:39.259474 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:39.310885 23730 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:39.312742 24521 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.312775 24258 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.313133 27614 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.313288 24390 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.313417 25808 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.314059 23994 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.314316 27939 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.314463 23864 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.314518 27614 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.314759 27940 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.315138 27940 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.315690 27614 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.317570 27614 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.319514 24390 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.319701 24521 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.319921 23994 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.320037 24258 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.322223 23588 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
W20240501 14:10:39.322546 23665 consensus_peers.cc:485] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d25ef07189ac43749efc549299dc89f8. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.323182 25808 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.324505 23864 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.332046 24128 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.333647 23994 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.333668 25940 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.333891 23730 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.334069 24258 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.334085 24521 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.334268 27461 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.334444 23864 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.334558 27963 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.334689 27461 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.334854 27962 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 14:10:39.335135 24061 consensus_peers.cc:485] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fee07797b5c84ad8aa71d10c1ac24fe1. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.335147 27461 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20240501 14:10:39.336432 27962 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.337867 27461 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.338527 24521 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.339000 24258 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.339026 23864 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.341065 23994 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.341431 23590 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.341583 25940 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.343557 23730 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.351253 24390 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:39.352972 24128 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.353097 26071 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.353263 23730 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.353318 27800 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.353381 23994 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 14:10:39.353752 24325 consensus_peers.cc:485] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fccb18388fb244bcb1dfd85908c0c118. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.353785 24521 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.353845 27800 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.354004 27823 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.354204 27980 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.355141 27822 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.355945 23864 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.356477 27800 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.356870 27800 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.357522 23730 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.357719 26071 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.359236 24521 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.359994 23581 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.360136 23864 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.360630 24128 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.365841 23994 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.371263 24390 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:39.373009 24258 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.373118 24521 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.373368 25940 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.373402 23864 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.373394 23994 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.373548 27981 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.373826 27981 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.373984 23730 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.374125 27981 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.374405 27980 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.375005 27982 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 14:10:39.375465 24325 consensus_peers.cc:485] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a50dc90c18174aa289c75056c4c461b0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.375602 27982 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.377647 27822 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.377827 24258 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.378165 23864 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.379757 25940 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.379931 23587 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:39.379990 23730 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.381893 23994 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.383744 24521 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.393023 24521 consensus_queue.cc:237] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.395435 26071 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.395594 24128 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.395905 23864 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.396191 24390 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.396203 23730 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.396706 26861 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.396893 28013 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.397078 28011 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.397239 26861 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.397363 28010 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.398131 23994 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.398555 28012 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.401194 28009 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
W20240501 14:10:39.403080 24457 consensus_peers.cc:485] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 182d85ec9c03441fb974ee50ad36ea8b. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.403873 23589 catalog_manager.cc:5484] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:39.404004 24128 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.403163 23994 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.406029 23730 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.407011 23864 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.407567 26071 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.408154 24390 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.418417 24390 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:39.419907 23730 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 14:10:39.419950 24325 consensus_peers.cc:485] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 84da4677d89b42fe8c2418cd156c75ae. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.420010 24258 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.420087 23994 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.420360 23864 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.420387 27823 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.420584 27800 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.420969 26071 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.421001 27982 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.421044 24521 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.421314 27982 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.421451 27980 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.422533 27981 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.423043 27980 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.423781 23730 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.424863 23864 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.425915 24258 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.426285 23589 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:39.426854 24521 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.428040 26071 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.428735 23994 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.438482 23994 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:39.440265 24390 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.440284 24128 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 14:10:39.440378 23929 consensus_peers.cc:485] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fc754b4d0f144c4ba0bd9a1b77f7bb4. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.440562 28035 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.440685 25808 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.441004 23730 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.441059 28035 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.441524 23864 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.441771 28034 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.441984 28035 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.442178 27902 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.442409 24521 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.442734 27902 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.444085 28035 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.444480 24390 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.445119 25808 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.446797 23582 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.447607 24521 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.448709 23730 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.449424 23864 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.451957 24128 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.457206 23730 consensus_queue.cc:237] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.458931 25940 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.459075 23994 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 14:10:39.459118 23665 consensus_peers.cc:485] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba3621a3f5df43eca96826da75fc31d0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.459228 24390 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.459360 24521 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.459440 27940 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.459602 27614 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.459735 27951 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.459793 23864 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.460175 27614 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.460465 27951 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.461313 24258 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.461727 27614 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.462438 27950 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.463817 23864 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.464008 24258 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.465058 24521 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.465313 23589 catalog_manager.cc:5484] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:39.467059 23994 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.467906 25940 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.470144 24390 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:39.477620 24390 consensus_queue.cc:237] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:39.479480 26071 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.479530 23730 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.479580 23994 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.479549 24128 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.479893 27982 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.480082 27800 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.480191 24258 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.480355 28026 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.480401 23864 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.480620 27800 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 14:10:39.481498 24325 consensus_peers.cc:485] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 777ba3b6be404d12bb0b219c66da71a6. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.481545 28025 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.481913 28025 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.482945 27800 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.483505 23994 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.483986 23730 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.485054 26071 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.485474 24128 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.488281 23589 catalog_manager.cc:5484] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:39.492182 23864 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.492234 24258 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:39.501967 23730 consensus_queue.cc:237] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:39.503728 24260 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.503926 23994 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.503916 24521 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.504022 25940 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.504256 27940 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.504449 27951 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.504681 27939 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.504942 24390 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 14:10:39.505033 23665 consensus_peers.cc:485] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37fda65c1f5240ce9f886d57921a03f0. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:39.505470 27951 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.505959 23864 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:39.506214 27940 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.507264 27951 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:39.508067 27940 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.508773 25940 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.509064 23994 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.511363 23588 catalog_manager.cc:5484] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.511297 24260 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.511685 24521 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.513911 24390 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.514441 23864 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:39.844432 28178 ts_tablet_manager.cc:902] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:39.844872 28178 tablet_copy_client.cc:308] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:46669
I20240501 14:10:39.845300 24148 tablet_copy_service.cc:140] P 166d7170d98944e88de2eccda9d30545: Received BeginTabletCopySession request for tablet fee07797b5c84ad8aa71d10c1ac24fe1 from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:41233)
I20240501 14:10:39.845393 24148 tablet_copy_service.cc:161] P 166d7170d98944e88de2eccda9d30545: Beginning new tablet copy session on tablet fee07797b5c84ad8aa71d10c1ac24fe1 from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:41233: session id = a308c0ed71f44f57967c97239d1198f7-fee07797b5c84ad8aa71d10c1ac24fe1
I20240501 14:10:39.846139 24148 tablet_copy_source_session.cc:205] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:39.847002 28178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:39.848855 28178 tablet_copy_client.cc:783] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:39.849092 28178 tablet_copy_client.cc:647] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:39.850064 28178 tablet_copy_client.cc:523] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:39.851011 28178 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:39.870182 28178 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:39.870417 28178 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:39.870559 28178 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.004s
I20240501 14:10:39.870918 28178 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.871026 28178 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:39.871117 28178 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.871393 28178 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:10:39.871829 24148 tablet_copy_service.cc:342] P 166d7170d98944e88de2eccda9d30545: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-fee07797b5c84ad8aa71d10c1ac24fe1 received from {username='slave'} at 127.6.153.201:41233
I20240501 14:10:39.871898 24148 tablet_copy_service.cc:433] P 166d7170d98944e88de2eccda9d30545: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-fee07797b5c84ad8aa71d10c1ac24fe1 on tablet fee07797b5c84ad8aa71d10c1ac24fe1 with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:39.887229 28185 ts_tablet_manager.cc:902] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:39.887670 28187 ts_tablet_manager.cc:902] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:39.887857 28185 tablet_copy_client.cc:308] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:39.888178 28187 tablet_copy_client.cc:308] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:39.888289 24412 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet 84da4677d89b42fe8c2418cd156c75ae from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:59269)
I20240501 14:10:39.888378 24412 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet 84da4677d89b42fe8c2418cd156c75ae from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:59269: session id = c958143102d04279ac25aa56214d0d4b-84da4677d89b42fe8c2418cd156c75ae
I20240501 14:10:39.889029 24411 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet fccb18388fb244bcb1dfd85908c0c118 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:59269)
I20240501 14:10:39.889065 24412 tablet_copy_source_session.cc:205] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:39.889108 24411 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet fccb18388fb244bcb1dfd85908c0c118 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:59269: session id = c958143102d04279ac25aa56214d0d4b-fccb18388fb244bcb1dfd85908c0c118
I20240501 14:10:39.889842 28185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:39.890015 24411 tablet_copy_source_session.cc:205] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:39.890813 28187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:39.891692 28185 tablet_copy_client.cc:783] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:39.891831 28185 tablet_copy_client.cc:647] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:39.892470 28187 tablet_copy_client.cc:783] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:39.892601 28187 tablet_copy_client.cc:647] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:39.893493 28187 tablet_copy_client.cc:523] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:39.894519 28187 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:39.895604 28185 tablet_copy_client.cc:523] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:39.896464 28185 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:39.908612 28178 ts_tablet_manager.cc:902] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:39.909009 28178 tablet_copy_client.cc:308] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:39.909464 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet ba3621a3f5df43eca96826da75fc31d0 from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:56835)
I20240501 14:10:39.909561 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet ba3621a3f5df43eca96826da75fc31d0 from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:56835: session id = a308c0ed71f44f57967c97239d1198f7-ba3621a3f5df43eca96826da75fc31d0
I20240501 14:10:39.910446 23752 tablet_copy_source_session.cc:205] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:39.911697 28178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3621a3f5df43eca96826da75fc31d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:39.913684 28178 tablet_copy_client.cc:783] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:39.913808 28178 tablet_copy_client.cc:647] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:39.917145 28178 tablet_copy_client.cc:523] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:39.918206 28178 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:39.923810 28187 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:39.924067 28187 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:39.924222 28187 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.030s user 0.021s sys 0.005s
I20240501 14:10:39.924597 28187 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:39.924728 28187 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:39.925073 28187 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:39.925493 28187 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 14:10:39.926052 24411 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-fccb18388fb244bcb1dfd85908c0c118 received from {username='slave'} at 127.6.153.202:59269
I20240501 14:10:39.926124 24411 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session c958143102d04279ac25aa56214d0d4b-fccb18388fb244bcb1dfd85908c0c118 on tablet fccb18388fb244bcb1dfd85908c0c118 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:39.928597 28185 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:39.928855 28185 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:39.929011 28185 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.002s
I20240501 14:10:39.929387 28185 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:39.929530 28185 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:39.929631 28185 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:39.929960 28185 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:39.930351 24411 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-84da4677d89b42fe8c2418cd156c75ae received from {username='slave'} at 127.6.153.202:59269
I20240501 14:10:39.930423 24411 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session c958143102d04279ac25aa56214d0d4b-84da4677d89b42fe8c2418cd156c75ae on tablet 84da4677d89b42fe8c2418cd156c75ae with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:39.931512 28202 ts_tablet_manager.cc:902] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:39.932968 28202 tablet_copy_client.cc:308] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:39.934587 28208 ts_tablet_manager.cc:902] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305)
I20240501 14:10:39.935057 28208 tablet_copy_client.cc:308] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.199:35305
I20240501 14:10:39.937494 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet d25ef07189ac43749efc549299dc89f8 from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:58017)
I20240501 14:10:39.937582 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet d25ef07189ac43749efc549299dc89f8 from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:58017: session id = 6d832f3f723641778e60b3fd855ff98e-d25ef07189ac43749efc549299dc89f8
I20240501 14:10:39.938464 23752 tablet_copy_source_session.cc:205] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:39.939359 28202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:39.941684 28202 tablet_copy_client.cc:783] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:39.941813 28202 tablet_copy_client.cc:647] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:39.942159 24544 tablet_copy_service.cc:140] P 40b69f74de5a447f86c1445f8e84850d: Received BeginTabletCopySession request for tablet 182d85ec9c03441fb974ee50ad36ea8b from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:51015)
I20240501 14:10:39.942276 24544 tablet_copy_service.cc:161] P 40b69f74de5a447f86c1445f8e84850d: Beginning new tablet copy session on tablet 182d85ec9c03441fb974ee50ad36ea8b from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:51015: session id = a308c0ed71f44f57967c97239d1198f7-182d85ec9c03441fb974ee50ad36ea8b
I20240501 14:10:39.943010 24544 tablet_copy_source_session.cc:205] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:39.943943 28208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 182d85ec9c03441fb974ee50ad36ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:39.944382 28202 tablet_copy_client.cc:523] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:39.945500 28202 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:39.945936 28208 tablet_copy_client.cc:783] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:39.946101 28208 tablet_copy_client.cc:647] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:39.947475 28208 tablet_copy_client.cc:523] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:39.948218 28178 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:39.949023 28208 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:39.963238 28202 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:39.963456 28202 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:39.963591 28202 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.004s
I20240501 14:10:39.963914 28202 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:39.964015 28202 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:39.964105 28202 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:39.964344 28202 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:39.964732 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-d25ef07189ac43749efc549299dc89f8 received from {username='slave'} at 127.6.153.200:58017
I20240501 14:10:39.964810 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-d25ef07189ac43749efc549299dc89f8 on tablet d25ef07189ac43749efc549299dc89f8 with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:39.971498 28178 tablet_bootstrap.cc:492] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:39.971659 28178 ts_tablet_manager.cc:1367] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.053s user 0.025s sys 0.000s
I20240501 14:10:39.972077 28178 raft_consensus.cc:348] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.972208 28178 raft_consensus.cc:729] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:39.972317 28178 consensus_queue.cc:260] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.972604 28178 ts_tablet_manager.cc:1398] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:39.973026 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-ba3621a3f5df43eca96826da75fc31d0 received from {username='slave'} at 127.6.153.201:56835
I20240501 14:10:39.973100 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-ba3621a3f5df43eca96826da75fc31d0 on tablet ba3621a3f5df43eca96826da75fc31d0 with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:39.986940 28187 ts_tablet_manager.cc:902] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:39.987404 28187 tablet_copy_client.cc:308] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:39.987931 28208 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:39.987955 24411 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet 777ba3b6be404d12bb0b219c66da71a6 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:59269)
I20240501 14:10:39.988061 24411 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet 777ba3b6be404d12bb0b219c66da71a6 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:59269: session id = c958143102d04279ac25aa56214d0d4b-777ba3b6be404d12bb0b219c66da71a6
I20240501 14:10:39.988080 28202 ts_tablet_manager.cc:902] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:39.988155 28208 tablet_bootstrap.cc:492] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:39.988294 28208 ts_tablet_manager.cc:1367] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.039s user 0.014s sys 0.005s
I20240501 14:10:39.988469 28202 tablet_copy_client.cc:308] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:39.988674 28208 raft_consensus.cc:348] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.988781 28208 raft_consensus.cc:729] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:39.988879 24411 tablet_copy_source_session.cc:205] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:39.988881 24412 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet a50dc90c18174aa289c75056c4c461b0 from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:41349)
I20240501 14:10:39.988875 28208 consensus_queue.cc:260] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:39.988992 24412 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet a50dc90c18174aa289c75056c4c461b0 from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:41349: session id = 6d832f3f723641778e60b3fd855ff98e-a50dc90c18174aa289c75056c4c461b0
I20240501 14:10:39.989180 28208 ts_tablet_manager.cc:1398] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:39.989544 24544 tablet_copy_service.cc:342] P 40b69f74de5a447f86c1445f8e84850d: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-182d85ec9c03441fb974ee50ad36ea8b received from {username='slave'} at 127.6.153.201:51015
I20240501 14:10:39.989615 24544 tablet_copy_service.cc:433] P 40b69f74de5a447f86c1445f8e84850d: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-182d85ec9c03441fb974ee50ad36ea8b on tablet 182d85ec9c03441fb974ee50ad36ea8b with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:39.989804 24412 tablet_copy_source_session.cc:205] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:39.989928 28187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777ba3b6be404d12bb0b219c66da71a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:39.990732 28202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:39.991976 28187 tablet_copy_client.cc:783] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:39.992100 28187 tablet_copy_client.cc:647] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:39.992460 28202 tablet_copy_client.cc:783] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:39.992581 28202 tablet_copy_client.cc:647] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:39.993584 28202 tablet_copy_client.cc:523] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:39.993640 28187 tablet_copy_client.cc:523] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:39.994581 28202 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:39.994583 28187 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:40.019827 28202 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:40.020089 28202 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:40.020258 28202 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.026s user 0.025s sys 0.000s
I20240501 14:10:40.020722 28202 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:40.020891 28202 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:40.021096 28187 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:40.021035 28202 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:40.021575 28187 tablet_bootstrap.cc:492] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:40.021766 28187 ts_tablet_manager.cc:1367] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.027s user 0.024s sys 0.003s
I20240501 14:10:40.022109 28202 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 14:10:40.022440 28187 raft_consensus.cc:348] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:40.022574 28187 raft_consensus.cc:729] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:40.022671 28187 consensus_queue.cc:260] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:40.022940 28187 ts_tablet_manager.cc:1398] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:40.023200 24412 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-a50dc90c18174aa289c75056c4c461b0 received from {username='slave'} at 127.6.153.200:41349
I20240501 14:10:40.023278 24412 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-a50dc90c18174aa289c75056c4c461b0 on tablet a50dc90c18174aa289c75056c4c461b0 with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:40.023847 24411 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-777ba3b6be404d12bb0b219c66da71a6 received from {username='slave'} at 127.6.153.202:59269
I20240501 14:10:40.023916 24411 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session c958143102d04279ac25aa56214d0d4b-777ba3b6be404d12bb0b219c66da71a6 on tablet 777ba3b6be404d12bb0b219c66da71a6 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:40.025281 28202 ts_tablet_manager.cc:902] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:40.025775 28202 tablet_copy_client.cc:308] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:40.026185 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet 37fda65c1f5240ce9f886d57921a03f0 from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:58017)
I20240501 14:10:40.026281 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet 37fda65c1f5240ce9f886d57921a03f0 from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:58017: session id = 6d832f3f723641778e60b3fd855ff98e-37fda65c1f5240ce9f886d57921a03f0
I20240501 14:10:40.026986 23752 tablet_copy_source_session.cc:205] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:40.027724 28202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fda65c1f5240ce9f886d57921a03f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:40.029373 28202 tablet_copy_client.cc:783] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:40.029492 28202 tablet_copy_client.cc:647] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:40.030547 28202 tablet_copy_client.cc:523] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:40.031491 28202 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:40.041962 28251 ts_tablet_manager.cc:902] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:40.042415 28251 tablet_copy_client.cc:308] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:44313
I20240501 14:10:40.042845 24016 tablet_copy_service.cc:140] P 8ebe0a098b0f4ed4861a5605717fecbd: Received BeginTabletCopySession request for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:49861)
I20240501 14:10:40.042940 24016 tablet_copy_service.cc:161] P 8ebe0a098b0f4ed4861a5605717fecbd: Beginning new tablet copy session on tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:49861: session id = 6d832f3f723641778e60b3fd855ff98e-0fc754b4d0f144c4ba0bd9a1b77f7bb4
I20240501 14:10:40.043808 24016 tablet_copy_source_session.cc:205] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:40.044545 28251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:40.046574 28251 tablet_copy_client.cc:783] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:40.047322 28251 tablet_copy_client.cc:647] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:40.048766 28251 tablet_copy_client.cc:523] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:40.049772 28251 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:40.054033 28202 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:40.071683 28202 tablet_bootstrap.cc:492] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:40.071848 28202 ts_tablet_manager.cc:1367] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.040s user 0.023s sys 0.000s
I20240501 14:10:40.072237 28202 raft_consensus.cc:348] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:40.072377 28202 raft_consensus.cc:729] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:40.072479 28202 consensus_queue.cc:260] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:40.072760 28202 ts_tablet_manager.cc:1398] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:40.076043 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-37fda65c1f5240ce9f886d57921a03f0 received from {username='slave'} at 127.6.153.200:58017
I20240501 14:10:40.076120 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-37fda65c1f5240ce9f886d57921a03f0 on tablet 37fda65c1f5240ce9f886d57921a03f0 with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:40.089125 28251 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:40.089355 28251 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:40.089498 28251 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.040s user 0.014s sys 0.004s
I20240501 14:10:40.089871 28251 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:40.089974 28251 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:40.090065 28251 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:40.090370 28251 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 14:10:40.090785 24016 tablet_copy_service.cc:342] P 8ebe0a098b0f4ed4861a5605717fecbd: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-0fc754b4d0f144c4ba0bd9a1b77f7bb4 received from {username='slave'} at 127.6.153.200:49861
I20240501 14:10:40.090866 24016 tablet_copy_service.cc:433] P 8ebe0a098b0f4ed4861a5605717fecbd: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-0fc754b4d0f144c4ba0bd9a1b77f7bb4 on tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:40.236915 25808 raft_consensus.cc:1203] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.293458 25940 raft_consensus.cc:1203] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.321194 25940 raft_consensus.cc:1203] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.375474 25808 raft_consensus.cc:1203] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.414251 26071 raft_consensus.cc:1203] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.431252 25808 raft_consensus.cc:1203] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.450976 26071 raft_consensus.cc:1203] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.513839 26071 raft_consensus.cc:1203] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.557662 26071 raft_consensus.cc:1203] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.597488 25940 raft_consensus.cc:1203] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:40.733943 27822 raft_consensus.cc:1050] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:40.734519 27822 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:40.736488 26071 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.736528 23994 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.736804 25940 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.736990 27800 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.737068 23730 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.737308 27800 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.737401 24521 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.737452 28300 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.737735 28397 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.737887 28300 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.739068 24258 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.739594 23862 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.739816 28398 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.739984 27800 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.742547 27800 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.743247 25940 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.744002 24258 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.744792 23862 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.745765 23730 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.746085 23584 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.746214 24521 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.748463 26071 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.750929 23994 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.754925 24390 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:40.756421 25940 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.756462 26071 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.756878 23730 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.756896 23862 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.757040 24521 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.757210 28300 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.757355 28398 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.757452 24258 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.757484 27800 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.757783 28300 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.758229 27800 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.758522 27800 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.759934 28398 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.760373 24521 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.760798 26071 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.762437 23730 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.762696 23586 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.763001 24258 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.763178 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 84da4677d89b42fe8c2418cd156c75ae with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:40.763878 23862 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.765408 25940 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.767621 23976 tablet_service.cc:1514] Processing DeleteTablet for tablet 84da4677d89b42fe8c2418cd156c75ae with delete_type TABLET_DATA_TOMBSTONED (TS 8ebe0a098b0f4ed4861a5605717fecbd not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:58422
I20240501 14:10:40.768641 28410 tablet_replica.cc:330] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: stopping tablet replica
I20240501 14:10:40.768956 28410 raft_consensus.cc:2229] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:40.769349 28410 raft_consensus.cc:2258] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:40.771113 28410 ts_tablet_manager.cc:1875] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:40.773303 28410 ts_tablet_manager.cc:1888] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:40.773393 28410 log.cc:1196] T 84da4677d89b42fe8c2418cd156c75ae P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/84da4677d89b42fe8c2418cd156c75ae
I20240501 14:10:40.773846 23577 catalog_manager.cc:4830] TS 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): tablet 84da4677d89b42fe8c2418cd156c75ae (table rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]) successfully deleted
I20240501 14:10:40.775120 28289 raft_consensus.cc:1050] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:40.775619 28289 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:40.777259 25940 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.777307 23730 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.777750 24521 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.777721 28413 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.777843 24258 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.777842 23994 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.778457 23862 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.778609 25808 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.778697 28289 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.778911 28411 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.779214 28289 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.779376 28411 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.779548 28413 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.779798 28411 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.782338 28414 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.783257 25940 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.783576 23994 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.784365 24521 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.784513 23862 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.784696 23730 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.786144 25808 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.787792 23588 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.789053 24258 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.794714 28343 raft_consensus.cc:1050] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:40.795233 28343 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:40.795437 24128 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:40.797245 25808 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.797533 24390 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.797849 24521 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.798202 25940 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.798213 25808 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.798514 26071 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.798527 28417 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.798847 28417 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.799101 23862 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.797713 28435 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.799587 23994 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.799726 28434 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.799773 23731 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.799963 27939 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.800274 28411 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.800416 23862 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.800603 28411 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.800798 24521 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.801071 28343 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.801379 28343 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.802039 24258 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.802379 28343 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.802664 28343 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.803086 28415 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.805740 28415 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.806036 28434 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.807509 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fee07797b5c84ad8aa71d10c1ac24fe1 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:40.807570 25808 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.807708 25793 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.808135 23731 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.809396 24524 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.809728 23582 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 reported cstate change: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.810493 24258 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.811161 28414 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.811731 23862 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.812247 24258 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.813052 23976 tablet_service.cc:1514] Processing DeleteTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 with delete_type TABLET_DATA_TOMBSTONED (TS 8ebe0a098b0f4ed4861a5605717fecbd not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:58422
I20240501 14:10:40.813246 28410 tablet_replica.cc:330] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: stopping tablet replica
I20240501 14:10:40.813159 25940 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.813367 28410 raft_consensus.cc:2229] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:40.813585 28410 raft_consensus.cc:2258] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:40.814600 28410 ts_tablet_manager.cc:1875] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:40.816121 23587 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 reported cstate change: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.816884 28410 ts_tablet_manager.cc:1888] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:40.816955 28410 log.cc:1196] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/fee07797b5c84ad8aa71d10c1ac24fe1
I20240501 14:10:40.818130 23577 catalog_manager.cc:4830] TS 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): tablet fee07797b5c84ad8aa71d10c1ac24fe1 (table rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]) successfully deleted
I20240501 14:10:40.819878 24524 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.819965 24258 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.820072 24390 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.822270 23994 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.822691 26071 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.823897 23862 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.825951 23731 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:40.827481 24524 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.827662 23994 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.827893 26071 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.827987 27939 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.828233 27942 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.828459 25793 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.828436 28343 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.828771 27942 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.828825 24258 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.829145 28343 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.829250 24390 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.829927 28343 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.830842 28343 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.831936 24521 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.832054 24390 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.832774 24258 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.833925 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d25ef07189ac43749efc549299dc89f8 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:40.833921 26071 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.833938 25793 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.834612 23586 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:40.837390 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet d25ef07189ac43749efc549299dc89f8 with delete_type TABLET_DATA_TOMBSTONED (TS 2502b9e8098b4bc5af7204835fce444b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33618
I20240501 14:10:40.838092 28448 tablet_replica.cc:330] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: stopping tablet replica
I20240501 14:10:40.838373 28448 raft_consensus.cc:2229] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:40.838785 28448 raft_consensus.cc:2258] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:40.839776 28448 ts_tablet_manager.cc:1875] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:40.839974 23994 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:40.842499 28448 ts_tablet_manager.cc:1888] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:40.842648 28448 log.cc:1196] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/d25ef07189ac43749efc549299dc89f8
I20240501 14:10:40.843415 23577 catalog_manager.cc:4830] TS 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): tablet d25ef07189ac43749efc549299dc89f8 (table rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]) successfully deleted
I20240501 14:10:40.851576 28432 raft_consensus.cc:1050] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:40.851992 28432 consensus_queue.cc:237] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:40.854094 24390 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.854133 25793 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.854396 28432 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.854476 24524 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.854595 25940 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.854887 23862 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.854930 28432 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.855427 24258 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.855751 27939 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.856003 28431 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.856987 23994 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.857010 28432 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.857328 28432 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.858544 28435 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.860662 25793 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.860258 28435 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.860989 23994 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.863200 23589 catalog_manager.cc:5484] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 reported cstate change: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.863512 24524 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.865061 23862 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.866050 24258 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.868654 25940 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.870069 24390 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.872960 23731 consensus_queue.cc:237] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:40.874522 23994 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.874752 25793 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.874919 24524 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.874917 24390 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.875254 24258 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.875320 27942 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.875617 27939 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.876267 27939 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.877377 28432 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.877496 25940 raft_consensus.cc:1261] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.877641 27942 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.877944 28433 consensus_queue.cc:1035] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.878224 27942 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.879750 25940 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.879760 24390 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.879840 23994 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.881508 25793 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.882611 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ba3621a3f5df43eca96826da75fc31d0 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:40.883272 24524 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.883960 23586 catalog_manager.cc:5484] T ba3621a3f5df43eca96826da75fc31d0 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.883977 24260 raft_consensus.cc:2932] T ba3621a3f5df43eca96826da75fc31d0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.887446 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet ba3621a3f5df43eca96826da75fc31d0 with delete_type TABLET_DATA_TOMBSTONED (TS 2502b9e8098b4bc5af7204835fce444b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33618
I20240501 14:10:40.887650 28448 tablet_replica.cc:330] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: stopping tablet replica
I20240501 14:10:40.887799 28448 raft_consensus.cc:2229] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:40.888053 28448 raft_consensus.cc:2258] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:40.888870 28448 ts_tablet_manager.cc:1875] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:40.890633 28448 ts_tablet_manager.cc:1888] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:40.890703 28448 log.cc:1196] T ba3621a3f5df43eca96826da75fc31d0 P 2502b9e8098b4bc5af7204835fce444b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/ba3621a3f5df43eca96826da75fc31d0
I20240501 14:10:40.891090 23577 catalog_manager.cc:4830] TS 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): tablet ba3621a3f5df43eca96826da75fc31d0 (table rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]) successfully deleted
I20240501 14:10:40.921583 23862 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:40.923370 23994 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:40.923707 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:40.923883 26071 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:40.924010 24392 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:40.924111 28506 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:40.924520 23731 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:40.924551 28507 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:40.924722 28505 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:40.924892 28184 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:40.925076 24524 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:40.925060 28504 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:40.925515 28510 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:40.928730 28507 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:40.929896 28398 raft_consensus.cc:1050] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:40.930353 24258 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:40.930423 24392 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:40.930389 28398 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:40.933044 26071 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:40.933836 23587 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:40.933863 24524 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:40.934646 23862 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.934787 23731 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:40.935891 23994 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.936482 24521 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.936237 28397 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.938558 23995 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:40.939033 28397 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.940856 24128 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.941131 28397 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.942333 28397 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.942646 26071 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.942847 25940 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.944295 28397 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.944470 28521 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20240501 14:10:40.945928 23797 consensus_peers.cc:485] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21c8e4484fc94645869c7a912df55e4c. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:40.946554 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.946427 28397 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.946959 28521 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.947039 26071 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.947945 23731 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.948701 24128 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.948930 23862 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.950165 25940 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.951048 23587 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e reported cstate change: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.951107 24521 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.951887 23995 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:40.957993 28342 raft_consensus.cc:1050] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:40.958459 28342 consensus_queue.cc:237] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:40.960175 25793 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.960516 23862 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.960549 24392 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.960614 23995 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.960906 23731 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.960980 28342 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.961082 24128 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.961119 28534 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.961431 28540 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.961582 28342 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.961807 26071 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:40.961910 28534 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.962622 28342 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.963457 28539 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:40.965363 28538 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.966550 23995 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.967484 26071 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.967734 23588 catalog_manager.cc:5484] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:40.968194 25793 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.968559 23731 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.968541 23862 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.971822 24392 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.974581 24128 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.975811 24521 consensus_queue.cc:237] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:40.977320 23995 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.977418 23731 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.977597 25793 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.977597 26071 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.977705 28539 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.977946 28539 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.978058 28342 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.978122 24392 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.978190 24128 raft_consensus.cc:1261] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:40.978327 28342 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.978509 28536 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.979205 28537 consensus_queue.cc:1035] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:40.981586 23995 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.981585 25793 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.981585 28536 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 40b69f74de5a447f86c1445f8e84850d [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.983775 23581 catalog_manager.cc:5484] T 182d85ec9c03441fb974ee50ad36ea8b P a308c0ed71f44f57967c97239d1198f7 reported cstate change: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.984371 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 182d85ec9c03441fb974ee50ad36ea8b with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:40.984354 26071 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.985566 24392 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.985903 23731 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.987753 24128 raft_consensus.cc:2932] T 182d85ec9c03441fb974ee50ad36ea8b P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:40.990005 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet 182d85ec9c03441fb974ee50ad36ea8b with delete_type TABLET_DATA_TOMBSTONED (TS 2502b9e8098b4bc5af7204835fce444b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33618
I20240501 14:10:40.990202 28448 tablet_replica.cc:330] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: stopping tablet replica
I20240501 14:10:40.990342 28448 raft_consensus.cc:2229] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:40.990556 28448 raft_consensus.cc:2258] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:40.991210 28448 ts_tablet_manager.cc:1875] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:40.993098 28448 ts_tablet_manager.cc:1888] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:40.993157 28448 log.cc:1196] T 182d85ec9c03441fb974ee50ad36ea8b P 2502b9e8098b4bc5af7204835fce444b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/182d85ec9c03441fb974ee50ad36ea8b
I20240501 14:10:40.993602 23577 catalog_manager.cc:4830] TS 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): tablet 182d85ec9c03441fb974ee50ad36ea8b (table rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]) successfully deleted
I20240501 14:10:41.022650 28589 raft_consensus.cc:1050] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:41.023149 28589 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:41.024725 24260 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:41.025435 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.025748 23862 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.025763 25793 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.025823 28594 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.026185 24521 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.027058 23731 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.027122 24128 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.027428 25793 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:41.027469 27902 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.027774 27902 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.028131 23731 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:41.028232 23995 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:41.028240 23862 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:41.028674 27902 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.028697 24521 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:41.028960 27902 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.029059 28599 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:41.030354 28589 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.030834 28599 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:41.031144 24390 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:41.031505 28599 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:41.031795 28599 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:41.027967 26071 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.032076 28599 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:41.032210 28593 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.032734 28214 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:41.034371 28593 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.035080 24392 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
W20240501 14:10:41.035765 24193 consensus_peers.cc:485] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3377f84deb6249668991e5a3f6d63876. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:41.036558 24521 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.036929 24128 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.037020 23730 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.037086 26071 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.038523 25793 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.039440 23864 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:41.039963 23584 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.040988 28214 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:41.042960 24392 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:41.043740 23730 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:41.045300 24524 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:41.045341 23864 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.045504 25808 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:41.047374 23586 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:41.049654 23995 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:41.049633 23994 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:41.051188 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.051232 26071 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.051245 23730 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.051587 28593 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.051673 24128 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.051893 28593 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.052019 24524 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.052260 28594 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.052644 28592 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.052925 28592 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.054782 28592 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.055264 24524 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.057013 24392 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.057212 23730 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.058126 23577 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:41.058096 24128 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.058887 25808 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.059206 28591 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.060189 25808 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.062225 23588 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:41.063752 26071 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.067519 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with delete_type TABLET_DATA_TOMBSTONED (TS 2502b9e8098b4bc5af7204835fce444b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33618
I20240501 14:10:41.067695 28448 tablet_replica.cc:330] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: stopping tablet replica
I20240501 14:10:41.067818 28448 raft_consensus.cc:2229] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:41.068015 28448 raft_consensus.cc:2258] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:41.071827 28448 ts_tablet_manager.cc:1875] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:41.073770 28448 ts_tablet_manager.cc:1888] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:41.073840 28448 log.cc:1196] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 2502b9e8098b4bc5af7204835fce444b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/0fc754b4d0f144c4ba0bd9a1b77f7bb4
I20240501 14:10:41.074273 23577 catalog_manager.cc:4830] TS 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 (table rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]) successfully deleted
I20240501 14:10:41.087486 28432 raft_consensus.cc:1050] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:41.087985 28432 consensus_queue.cc:237] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:41.089900 23864 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.089919 24524 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.090188 23995 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.090371 27939 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.090569 24260 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.090593 27942 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.091051 26071 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.091172 25940 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.091430 27939 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.091630 24392 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.092131 27939 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.093335 27939 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.093647 28434 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.093817 28431 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.095628 28434 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.096218 23995 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.096428 23864 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.096931 24392 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.099385 23587 catalog_manager.cc:5484] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.099722 24524 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.103605 26071 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.105798 24260 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.105842 25940 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.110707 23730 consensus_queue.cc:237] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:41.112349 23864 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.112358 26071 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.112640 24392 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.112677 25940 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.112810 28433 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.113011 24260 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.113158 28432 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.113420 27939 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.113556 28435 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.113971 24524 raft_consensus.cc:1261] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.114197 28432 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.114390 27942 consensus_queue.cc:1035] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.116428 28432 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.117450 24260 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.117942 24390 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.119477 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 37fda65c1f5240ce9f886d57921a03f0 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:41.119452 23584 catalog_manager.cc:5484] T 37fda65c1f5240ce9f886d57921a03f0 P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:41.120034 23864 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.120554 26071 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.120774 24524 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.121585 25940 raft_consensus.cc:2932] T 37fda65c1f5240ce9f886d57921a03f0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.125195 23976 tablet_service.cc:1514] Processing DeleteTablet for tablet 37fda65c1f5240ce9f886d57921a03f0 with delete_type TABLET_DATA_TOMBSTONED (TS 8ebe0a098b0f4ed4861a5605717fecbd not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:58422
I20240501 14:10:41.125409 28410 tablet_replica.cc:330] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: stopping tablet replica
I20240501 14:10:41.125535 28410 raft_consensus.cc:2229] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:41.125732 28410 raft_consensus.cc:2258] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:41.126399 28410 ts_tablet_manager.cc:1875] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:41.128265 28410 ts_tablet_manager.cc:1888] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:41.128345 28410 log.cc:1196] T 37fda65c1f5240ce9f886d57921a03f0 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/37fda65c1f5240ce9f886d57921a03f0
I20240501 14:10:41.128743 23577 catalog_manager.cc:4830] TS 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): tablet 37fda65c1f5240ce9f886d57921a03f0 (table rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]) successfully deleted
I20240501 14:10:41.163035 28519 raft_consensus.cc:1050] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:41.163580 28519 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:41.165446 24524 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.165769 25940 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.165766 24260 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.166147 27822 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.166326 28518 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.166406 26071 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.166947 28519 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.167299 28520 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.167606 23995 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.167809 23864 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.167995 28521 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.169209 23730 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.169528 28521 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.169687 27822 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.171411 28520 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.172133 23730 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.172138 23995 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.173282 25940 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.173605 24524 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.173713 26071 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.175247 24260 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.175756 23589 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.178332 23864 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.183754 24390 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:41.185139 25940 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.185407 24260 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.185429 23730 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.185424 26071 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.185528 28519 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.185775 27822 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.185914 23864 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.185931 28397 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.186056 28519 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.186533 28397 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.186769 24524 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.187498 28397 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.188953 28520 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.189374 26071 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.190088 23730 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.192054 25940 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.192430 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a50dc90c18174aa289c75056c4c461b0 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:41.192629 23590 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e reported cstate change: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.192785 24260 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.192836 24524 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.196420 23976 tablet_service.cc:1514] Processing DeleteTablet for tablet a50dc90c18174aa289c75056c4c461b0 with delete_type TABLET_DATA_TOMBSTONED (TS 8ebe0a098b0f4ed4861a5605717fecbd not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:58422
I20240501 14:10:41.196627 28410 tablet_replica.cc:330] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: stopping tablet replica
I20240501 14:10:41.196744 28410 raft_consensus.cc:2229] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:41.196921 28410 raft_consensus.cc:2258] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:41.197818 28410 ts_tablet_manager.cc:1875] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:41.199669 28410 ts_tablet_manager.cc:1888] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:41.199739 28410 log.cc:1196] T a50dc90c18174aa289c75056c4c461b0 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/a50dc90c18174aa289c75056c4c461b0
I20240501 14:10:41.199965 23864 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:41.200294 23577 catalog_manager.cc:4830] TS 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): tablet a50dc90c18174aa289c75056c4c461b0 (table rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]) successfully deleted
I20240501 14:10:41.201846 28519 raft_consensus.cc:1050] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:41.202312 28519 consensus_queue.cc:237] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:41.204346 24260 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.204496 23864 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.204718 23995 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.204962 23731 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.205066 25940 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.205077 28397 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.205370 26071 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.205404 28398 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.205955 28520 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.206221 28520 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.208187 28397 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.208454 27800 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.209030 27800 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.210000 24128 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:41.210667 28300 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:41.210995 24260 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.211268 23731 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.212432 25940 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.213661 23995 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.214195 23586 catalog_manager.cc:5484] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.214201 23864 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.214828 24128 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.218041 26071 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.301262 28751 ts_tablet_manager.cc:902] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427)
I20240501 14:10:41.301728 28751 tablet_copy_client.cc:308] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.194:34427
I20240501 14:10:41.305318 23884 tablet_copy_service.cc:140] P 2502b9e8098b4bc5af7204835fce444b: Received BeginTabletCopySession request for tablet 21c8e4484fc94645869c7a912df55e4c from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:37409)
I20240501 14:10:41.305424 23884 tablet_copy_service.cc:161] P 2502b9e8098b4bc5af7204835fce444b: Beginning new tablet copy session on tablet 21c8e4484fc94645869c7a912df55e4c from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:37409: session id = c958143102d04279ac25aa56214d0d4b-21c8e4484fc94645869c7a912df55e4c
I20240501 14:10:41.306278 23884 tablet_copy_source_session.cc:205] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:41.307073 28751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c8e4484fc94645869c7a912df55e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:41.308889 28751 tablet_copy_client.cc:783] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:41.309034 28751 tablet_copy_client.cc:647] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:41.310038 28751 tablet_copy_client.cc:523] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:41.311009 28751 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:41.351568 28751 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:41.351797 28751 tablet_bootstrap.cc:492] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:41.351953 28751 ts_tablet_manager.cc:1367] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.041s user 0.020s sys 0.000s
I20240501 14:10:41.352373 28751 raft_consensus.cc:348] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:41.352481 28751 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:41.352572 28751 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:41.352821 28751 ts_tablet_manager.cc:1398] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:41.353260 23884 tablet_copy_service.cc:342] P 2502b9e8098b4bc5af7204835fce444b: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-21c8e4484fc94645869c7a912df55e4c received from {username='slave'} at 127.6.153.202:37409
I20240501 14:10:41.353339 23884 tablet_copy_service.cc:433] P 2502b9e8098b4bc5af7204835fce444b: ending tablet copy session c958143102d04279ac25aa56214d0d4b-21c8e4484fc94645869c7a912df55e4c on tablet 21c8e4484fc94645869c7a912df55e4c with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:41.529832 24392 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:41.531524 26071 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.531548 25940 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.531881 24128 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.531911 28397 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.531996 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.532312 24524 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.532341 28397 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.533007 28398 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.533288 28398 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.534732 28398 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.535125 25940 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.535493 23994 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.535445 24524 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.536976 23731 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.537214 26071 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.537580 28520 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.538341 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fccb18388fb244bcb1dfd85908c0c118 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:41.539359 23582 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.541253 28521 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.541515 23994 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.543025 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet fccb18388fb244bcb1dfd85908c0c118 with delete_type TABLET_DATA_TOMBSTONED (TS 2502b9e8098b4bc5af7204835fce444b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33618
I20240501 14:10:41.543289 28448 tablet_replica.cc:330] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: stopping tablet replica
I20240501 14:10:41.543437 28448 raft_consensus.cc:2229] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:41.543678 28448 raft_consensus.cc:2258] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:41.544013 24128 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.544591 28448 ts_tablet_manager.cc:1875] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:41.546499 28448 ts_tablet_manager.cc:1888] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:41.546561 28448 log.cc:1196] T fccb18388fb244bcb1dfd85908c0c118 P 2502b9e8098b4bc5af7204835fce444b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/fccb18388fb244bcb1dfd85908c0c118
I20240501 14:10:41.547092 23577 catalog_manager.cc:4830] TS 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): tablet fccb18388fb244bcb1dfd85908c0c118 (table rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]) successfully deleted
I20240501 14:10:41.623293 28779 ts_tablet_manager.cc:902] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:41.623790 28779 tablet_copy_client.cc:308] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:38101
I20240501 14:10:41.626871 24280 tablet_copy_service.cc:140] P dbeb6a07fa7a44019ae00c169c414529: Received BeginTabletCopySession request for tablet 3377f84deb6249668991e5a3f6d63876 from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:47351)
I20240501 14:10:41.626979 24280 tablet_copy_service.cc:161] P dbeb6a07fa7a44019ae00c169c414529: Beginning new tablet copy session on tablet 3377f84deb6249668991e5a3f6d63876 from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:47351: session id = 6d832f3f723641778e60b3fd855ff98e-3377f84deb6249668991e5a3f6d63876
I20240501 14:10:41.627877 24280 tablet_copy_source_session.cc:205] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:41.628729 28779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:41.630371 28779 tablet_copy_client.cc:783] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:41.630511 28779 tablet_copy_client.cc:647] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:41.631487 28779 tablet_copy_client.cc:523] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:41.632375 28779 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:41.651429 28779 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:41.651710 28779 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:41.651895 28779 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.003s
I20240501 14:10:41.652258 28779 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:41.652367 28779 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:41.652444 28779 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:41.652688 28779 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:41.653138 24280 tablet_copy_service.cc:342] P dbeb6a07fa7a44019ae00c169c414529: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-3377f84deb6249668991e5a3f6d63876 received from {username='slave'} at 127.6.153.200:47351
I20240501 14:10:41.653224 24280 tablet_copy_service.cc:433] P dbeb6a07fa7a44019ae00c169c414529: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-3377f84deb6249668991e5a3f6d63876 on tablet 3377f84deb6249668991e5a3f6d63876 with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:41.681123 25940 raft_consensus.cc:1203] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:41.784353 24392 consensus_queue.cc:237] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:41.785944 26071 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.786274 23864 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.786348 28398 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.786705 28398 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.786752 24260 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.787017 25940 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.787755 23731 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.787925 24128 raft_consensus.cc:1261] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:41.788134 28398 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.788578 28397 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.788712 28521 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.788875 28520 consensus_queue.cc:1035] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:41.790143 28521 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.790534 24260 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.791745 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 777ba3b6be404d12bb0b219c66da71a6 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:41.792212 23731 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.792831 26071 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.793077 23589 catalog_manager.cc:5484] T 777ba3b6be404d12bb0b219c66da71a6 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.793766 23864 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.796160 23976 tablet_service.cc:1514] Processing DeleteTablet for tablet 777ba3b6be404d12bb0b219c66da71a6 with delete_type TABLET_DATA_TOMBSTONED (TS 8ebe0a098b0f4ed4861a5605717fecbd not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:58422
I20240501 14:10:41.796278 25940 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.796507 28821 tablet_replica.cc:330] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: stopping tablet replica
I20240501 14:10:41.796661 28821 raft_consensus.cc:2229] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:41.796919 28821 raft_consensus.cc:2258] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:41.797533 28821 ts_tablet_manager.cc:1875] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:41.798774 24128 raft_consensus.cc:2932] T 777ba3b6be404d12bb0b219c66da71a6 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:41.799875 28821 ts_tablet_manager.cc:1888] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:41.799948 28821 log.cc:1196] T 777ba3b6be404d12bb0b219c66da71a6 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/777ba3b6be404d12bb0b219c66da71a6
I20240501 14:10:41.800439 23577 catalog_manager.cc:4830] TS 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): tablet 777ba3b6be404d12bb0b219c66da71a6 (table rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]) successfully deleted
I20240501 14:10:42.140338 26071 raft_consensus.cc:1203] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:42.292542 28506 raft_consensus.cc:1050] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:42.293016 28506 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:42.294633 26071 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.294730 24524 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.295042 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.295130 23731 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.295223 24392 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.295462 28862 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.295954 25940 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.295657 28864 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.296080 23994 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 2502b9e8098b4bc5af7204835fce444b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.296543 28866 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.296840 28866 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.297835 28863 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.297972 28860 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.298435 28860 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.301693 28860 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.302791 26071 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.302791 23994 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.303864 24524 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.304864 24392 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.305768 25940 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.305785 24258 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.305828 23590 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e reported cstate change: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2502b9e8098b4bc5af7204835fce444b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.311460 23731 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:42.359583 23864 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c"
dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56660
I20240501 14:10:42.359752 23864 raft_consensus.cc:595] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:42.575521 23864 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c"
dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56678
I20240501 14:10:42.575639 23864 raft_consensus.cc:595] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:42.636071 28853 raft_consensus.cc:1050] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:42.636540 28853 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:42.638161 26071 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.638306 25793 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.638322 23731 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.638924 23864 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.638885 28822 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.639011 24524 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.639226 28822 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.639650 24392 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.639643 23994 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:42.639978 28909 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.640141 28911 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.640309 28596 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.640436 28822 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.641247 28853 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:42.643225 28853 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.643529 23864 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.645433 26071 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.645787 25793 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.647425 23581 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:42.647617 24392 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.649436 23731 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.651518 24524 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.652676 23994 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.655272 24258 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:42.656785 25793 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:42.656883 26071 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:42.657294 23994 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:42.657433 23731 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:42.657768 28912 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:42.657848 24392 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:42.657955 28911 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:42.658102 28822 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:42.658221 28853 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:42.658272 24524 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:42.658529 28853 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:42.658648 28596 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:42.661036 28912 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.661567 25793 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.661911 24392 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.662289 23731 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.662928 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3377f84deb6249668991e5a3f6d63876 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:42.663257 26071 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.663317 23994 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.665432 23582 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:42.665606 24524 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:42.668498 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet 3377f84deb6249668991e5a3f6d63876 with delete_type TABLET_DATA_TOMBSTONED (TS 2502b9e8098b4bc5af7204835fce444b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33618
I20240501 14:10:42.670809 28921 tablet_replica.cc:330] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: stopping tablet replica
I20240501 14:10:42.670945 28921 raft_consensus.cc:2229] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:42.671227 28921 raft_consensus.cc:2258] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:42.672439 28921 ts_tablet_manager.cc:1875] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:42.674041 28921 ts_tablet_manager.cc:1888] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:42.674121 28921 log.cc:1196] T 3377f84deb6249668991e5a3f6d63876 P 2502b9e8098b4bc5af7204835fce444b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/3377f84deb6249668991e5a3f6d63876
I20240501 14:10:42.674554 23577 catalog_manager.cc:4830] TS 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): tablet 3377f84deb6249668991e5a3f6d63876 (table rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]) successfully deleted
I20240501 14:10:42.719055 28863 raft_consensus.cc:979] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: : Instructing follower 40b69f74de5a447f86c1445f8e84850d to start an election
I20240501 14:10:42.719167 28863 raft_consensus.cc:1067] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Signalling peer 40b69f74de5a447f86c1445f8e84850d to start an election
I20240501 14:10:42.719740 24524 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c"
dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
from {username='slave'} at 127.6.153.194:35319
I20240501 14:10:42.719935 24524 raft_consensus.cc:482] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:10:42.720002 24524 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:42.720990 24524 raft_consensus.cc:504] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:42.722049 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:42.722090 24258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:42.722168 24392 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:42.722218 24258 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:42.722400 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:42.722512 23994 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:42.723035 24258 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 2.
I20240501 14:10:42.723119 24392 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 2.
I20240501 14:10:42.723302 23731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:42.723404 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:42.723445 23731 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:42.723520 23864 raft_consensus.cc:3032] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:10:42.723547 23994 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 2.
I20240501 14:10:42.723572 23864 raft_consensus.cc:729] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Running, Role: LEADER
I20240501 14:10:42.723737 23864 consensus_queue.cc:260] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:42.724012 23864 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:42.724335 23731 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 2.
I20240501 14:10:42.724735 23864 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 2.
I20240501 14:10:42.724759 24524 leader_election.cc:288] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [CANDIDATE]: Term 2 election: Requested vote from peers dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319), c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929)
I20240501 14:10:42.724999 26071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "6d832f3f723641778e60b3fd855ff98e"
I20240501 14:10:42.725173 26071 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:42.725184 24457 leader_election.cc:302] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 8 voters: 5 yes votes; 0 no votes. yes voters: 24d90b55560e45feb5fe4b9482317a4c, 40b69f74de5a447f86c1445f8e84850d, 7e6e9293a4614dabb5ad89fe3f4d5d04, 8ebe0a098b0f4ed4861a5605717fecbd, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:42.725344 28536 raft_consensus.cc:2781] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:10:42.725481 28536 raft_consensus.cc:686] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 2 LEADER]: Becoming Leader. State: Replica: 40b69f74de5a447f86c1445f8e84850d, State: Running, Role: LEADER
I20240501 14:10:42.725630 28536 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:42.727880 23584 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d reported cstate change: term changed from 1 to 2, leader changed from 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) to 40b69f74de5a447f86c1445f8e84850d (127.6.153.199). New cstate: current_term: 2 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:42.728191 26071 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 2.
I20240501 14:10:42.730346 25940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c8e4484fc94645869c7a912df55e4c" candidate_uuid: "40b69f74de5a447f86c1445f8e84850d" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "c958143102d04279ac25aa56214d0d4b"
I20240501 14:10:42.730502 25940 raft_consensus.cc:3037] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:42.731287 25940 raft_consensus.cc:2454] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40b69f74de5a447f86c1445f8e84850d in term 2.
I20240501 14:10:43.110242 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:43.110587 28924 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:43.163859 24392 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:43.164229 28924 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:43.203032 25940 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:43.203378 28924 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:43.212278 23731 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:43.212612 28924 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:43.215744 23994 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:43.215771 23864 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:43.216552 28955 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:43.217396 26071 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:43.217944 28924 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:43.219745 28957 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20240501 14:10:43.735687 24521 consensus_queue.cc:237] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 9, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:43.736990 26071 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:43.736996 25940 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:43.737795 23731 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:43.737849 23994 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:43.737874 24258 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:43.738284 28955 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.738566 28955 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.738802 28957 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.738922 28956 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.738937 24390 raft_consensus.cc:1261] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 2 FOLLOWER]: Refusing update from remote peer 40b69f74de5a447f86c1445f8e84850d: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:43.740037 28956 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.742651 28956 consensus_queue.cc:1035] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.742970 28957 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 40b69f74de5a447f86c1445f8e84850d [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:43.743640 23994 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:43.744638 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 21c8e4484fc94645869c7a912df55e4c with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:43.745039 24390 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:43.745133 23731 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 24d90b55560e45feb5fe4b9482317a4c [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:43.746953 23584 catalog_manager.cc:5484] T 21c8e4484fc94645869c7a912df55e4c P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 9 to 11, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New cstate: current_term: 2 leader_uuid: "40b69f74de5a447f86c1445f8e84850d" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:43.747561 26071 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P 6d832f3f723641778e60b3fd855ff98e [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:43.747561 25940 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P c958143102d04279ac25aa56214d0d4b [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:43.750514 24258 raft_consensus.cc:2932] T 21c8e4484fc94645869c7a912df55e4c P dbeb6a07fa7a44019ae00c169c414529 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:43.753777 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet 21c8e4484fc94645869c7a912df55e4c with delete_type TABLET_DATA_TOMBSTONED (TS 2502b9e8098b4bc5af7204835fce444b not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:33618
I20240501 14:10:43.754137 28984 tablet_replica.cc:330] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: stopping tablet replica
I20240501 14:10:43.754289 28984 raft_consensus.cc:2229] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:43.754411 28984 raft_consensus.cc:2258] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:43.755532 28984 ts_tablet_manager.cc:1875] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:43.757206 28984 ts_tablet_manager.cc:1888] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.10
I20240501 14:10:43.757267 28984 log.cc:1196] T 21c8e4484fc94645869c7a912df55e4c P 2502b9e8098b4bc5af7204835fce444b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/21c8e4484fc94645869c7a912df55e4c
I20240501 14:10:43.757694 23577 catalog_manager.cc:4830] TS 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): tablet 21c8e4484fc94645869c7a912df55e4c (table rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]) successfully deleted
I20240501 14:10:43.885687 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:43.886730 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:43.895511 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:43.895782 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:43.903512 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:43.912241 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:43.914450 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:43.921329 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:43.927785 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:43.939874 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:43.983265 24390 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:43.985427 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:43.985447 23994 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:43.986173 25940 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:43.986261 26071 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:43.986202 29045 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.986461 29038 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.986796 29038 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.986995 29047 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.987324 24521 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:43.987550 24128 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:43.988322 28519 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:43.988816 29045 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 14:10:43.990350 24325 consensus_peers.cc:485] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fccb18388fb244bcb1dfd85908c0c118. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:43.990571 28519 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:43.991115 24521 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:43.992120 25940 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:43.992130 26071 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:43.993661 23995 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:43.994174 23582 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 10 to 11, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:43.995041 23731 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:43.999310 24127 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:44.503382 29076 ts_tablet_manager.cc:902] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:44.503851 29076 tablet_copy_client.cc:308] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:44.506927 24411 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet fccb18388fb244bcb1dfd85908c0c118 from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:32905)
I20240501 14:10:44.507014 24411 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet fccb18388fb244bcb1dfd85908c0c118 from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:32905: session id = a308c0ed71f44f57967c97239d1198f7-fccb18388fb244bcb1dfd85908c0c118
I20240501 14:10:44.507753 24411 tablet_copy_source_session.cc:205] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:44.508531 29076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fccb18388fb244bcb1dfd85908c0c118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:44.510386 29076 tablet_copy_client.cc:783] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:44.510516 29076 tablet_copy_client.cc:647] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:44.511488 29076 tablet_copy_client.cc:523] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:44.512398 29076 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:44.531400 29076 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:44.531606 29076 tablet_bootstrap.cc:492] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:44.531759 29076 ts_tablet_manager.cc:1367] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.005s
I20240501 14:10:44.532166 29076 raft_consensus.cc:348] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:44.532294 29076 raft_consensus.cc:729] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:44.532380 29076 consensus_queue.cc:260] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:44.532737 29076 ts_tablet_manager.cc:1398] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:44.533149 24411 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-fccb18388fb244bcb1dfd85908c0c118 received from {username='slave'} at 127.6.153.201:32905
I20240501 14:10:44.533222 24411 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-fccb18388fb244bcb1dfd85908c0c118 on tablet fccb18388fb244bcb1dfd85908c0c118 with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:44.914585 25808 raft_consensus.cc:1203] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 14:10:45.362476 29045 raft_consensus.cc:1050] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:45.362941 29045 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:45.364681 25940 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.364948 25808 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.364974 26071 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.365339 29075 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.365592 24521 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.365577 29046 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.365718 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.365973 29045 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.366261 29107 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.366494 24127 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.366551 23995 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.366822 29045 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.366963 29110 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.368008 29046 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.370138 29075 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.371207 24127 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.371255 26071 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.371519 23995 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.373523 23731 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.374246 23589 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.375030 25940 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.375209 24521 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.376127 25808 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.381786 24390 consensus_queue.cc:237] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:45.383450 25808 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:45.383500 25940 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:45.383860 26071 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:45.383939 23731 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:45.383915 29046 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:45.384241 24524 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:45.384287 24127 raft_consensus.cc:1261] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:45.384264 29046 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:45.384754 29046 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:45.385975 29046 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:45.386272 29046 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:45.386564 29046 consensus_queue.cc:1035] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:45.387807 29046 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.388235 23731 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.389246 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fccb18388fb244bcb1dfd85908c0c118 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:45.390025 24524 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.390288 23588 catalog_manager.cc:5484] T fccb18388fb244bcb1dfd85908c0c118 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 12 to 13, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:45.390584 25808 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.390597 26071 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.392868 24127 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.393781 25940 raft_consensus.cc:2932] T fccb18388fb244bcb1dfd85908c0c118 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:45.396847 23976 tablet_service.cc:1514] Processing DeleteTablet for tablet fccb18388fb244bcb1dfd85908c0c118 with delete_type TABLET_DATA_TOMBSTONED (TS 8ebe0a098b0f4ed4861a5605717fecbd not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:58422
I20240501 14:10:45.397275 29126 tablet_replica.cc:330] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: stopping tablet replica
I20240501 14:10:45.397568 29126 raft_consensus.cc:2229] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:45.397799 29126 raft_consensus.cc:2258] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:45.398998 29126 ts_tablet_manager.cc:1875] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:45.400977 29126 ts_tablet_manager.cc:1888] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 14:10:45.401037 29126 log.cc:1196] T fccb18388fb244bcb1dfd85908c0c118 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/fccb18388fb244bcb1dfd85908c0c118
I20240501 14:10:45.401459 23577 catalog_manager.cc:4830] TS 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): tablet fccb18388fb244bcb1dfd85908c0c118 (table rebalance_test_table_3 [id=70240d945b994387bf27dafd724be678]) successfully deleted
I20240501 14:10:45.505410 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:45.509790 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:45.512795 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:45.518473 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:45.525596 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:45.526348 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:45.529156 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:45.531440 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:45.559463 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:45.562078 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:45.677557 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:45.683019 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:45.684945 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:45.685703 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:45.687876 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:45.701238 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:45.701880 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:45.712020 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:45.712218 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:45.712481 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:45.776909 23730 consensus_queue.cc:237] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:45.777139 23731 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:45.779196 24127 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } }
I20240501 14:10:45.779570 25940 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.780171 23995 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.780508 24260 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.780696 24524 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.780891 29119 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.781009 24126 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.781355 23864 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.781908 24524 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.782097 24260 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 14:10:45.782135 23665 consensus_peers.cc:485] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 697668f5caf34e63a9e5953c180959ee. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.782266 28758 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.782579 25808 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.782783 24390 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.782945 25940 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.783056 24392 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.783233 23995 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.783349 26071 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.783430 28758 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.783610 29119 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.783704 28758 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.783927 29119 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 14:10:45.784027 23667 consensus_peers.cc:485] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c -> Peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): Couldn't send request to peer 166d7170d98944e88de2eccda9d30545. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.784073 24521 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.784194 29119 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.784446 25808 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.784482 29119 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.784744 29119 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.785024 29119 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.785171 29105 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.785290 29119 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.785526 29119 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.785799 29119 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.786020 23864 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.786057 29119 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 14:10:45.786512 24064 consensus_peers.cc:485] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 -> Peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): Couldn't send request to peer 8ebe0a098b0f4ed4861a5605717fecbd. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.786800 29105 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.786345 29119 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20240501 14:10:45.787613 23731 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.789379 28758 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.792781 29265 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } } }
I20240501 14:10:45.792997 29119 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:45.793812 23864 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } } }
I20240501 14:10:45.794229 25808 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } } }
I20240501 14:10:45.795228 24392 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:45.795542 24521 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } } }
I20240501 14:10:45.796541 23581 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 10 to 11, NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) added. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } } }
I20240501 14:10:45.797648 24258 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } } }
I20240501 14:10:45.798817 29119 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.798872 25941 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:45.799520 23588 catalog_manager.cc:5484] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:45.800815 24126 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:45.802119 24522 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.802368 24521 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:45.803836 26071 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.803880 24260 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.805974 23994 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:45.807492 23995 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.810099 23582 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e reported cstate change: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.810546 23862 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:45.811441 24392 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.812134 25793 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.814752 23730 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } } }
I20240501 14:10:45.817195 25941 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } } }
I20240501 14:10:45.826526 24260 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:45.826531 23730 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } }
W20240501 14:10:45.828361 23667 consensus_peers.cc:485] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c -> Peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): Couldn't send request to peer 166d7170d98944e88de2eccda9d30545. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.828748 23995 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.828867 24521 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.828954 24522 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.829200 28909 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.829509 28909 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.829532 26071 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.829555 24392 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 14:10:45.829557 23668 consensus_peers.cc:485] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c -> Peer 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): Couldn't send request to peer 2502b9e8098b4bc5af7204835fce444b. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.829831 25793 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.830058 23995 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.830299 24126 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } }
I20240501 14:10:45.830614 29253 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.830874 29305 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.830957 29257 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.831132 23730 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.831506 29305 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.831573 24260 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.831915 29255 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.832800 24521 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.832876 25793 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.832968 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.833173 28758 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.833309 29105 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 14:10:45.833375 24193 consensus_peers.cc:485] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3377f84deb6249668991e5a3f6d63876. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.833541 29305 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.833585 29105 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.834125 23862 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.834554 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.835078 28758 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.835397 28758 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.835723 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.836087 29303 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 14:10:45.836997 24062 consensus_peers.cc:485] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 -> Peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): Couldn't send request to peer dbeb6a07fa7a44019ae00c169c414529. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.837515 29262 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } } }
I20240501 14:10:45.837884 26072 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.838002 24390 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.838156 29304 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.838421 25793 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:45.838521 23730 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.839733 23995 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } } }
I20240501 14:10:45.840088 29062 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.840443 29062 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.840327 23993 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.840677 29062 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:45.841761 23587 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 10 to 11, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:45.841845 24390 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } } }
I20240501 14:10:45.842870 23588 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 11 to 12, NON_VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) added. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } } }
I20240501 14:10:45.843986 25793 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } } }
I20240501 14:10:45.844700 29312 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.844763 26072 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } } }
I20240501 14:10:45.845181 24260 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } } }
I20240501 14:10:45.847802 24521 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } } }
I20240501 14:10:45.848016 29268 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } }
I20240501 14:10:45.849915 25793 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } }
I20240501 14:10:45.851095 24522 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.852224 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } }
I20240501 14:10:45.852504 23588 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 7 to 8, NON_VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) added. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:45.853832 23994 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } }
I20240501 14:10:45.855152 24126 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:45.856685 26071 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.856912 24392 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.858479 25795 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
W20240501 14:10:45.858539 24061 consensus_peers.cc:485] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de25122a01e845d2b35e4a1023789d3e. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.858815 29266 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:45.861284 24521 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } }
I20240501 14:10:45.862905 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:45.863234 29268 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20240501 14:10:45.863420 24062 consensus_peers.cc:485] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 -> Peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101): Couldn't send request to peer dbeb6a07fa7a44019ae00c169c414529. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.863476 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:45.863889 29268 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:45.865446 24522 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:45.865978 29268 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:45.867380 29266 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.867769 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:45.868259 29265 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.867852 24522 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.868918 23731 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } }
I20240501 14:10:45.869652 23864 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:45.870316 23731 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.870479 23587 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 8 to 9, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:45.870702 25808 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.872738 29265 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.873611 23994 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.873801 23864 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } }
I20240501 14:10:45.875202 23864 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.878437 25808 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.878809 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:45.884690 24260 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:45.886708 23731 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.886848 25941 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.886860 24392 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.887161 29304 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.887681 24522 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.887773 23994 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.888128 29306 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.888386 23864 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 14:10:45.888736 24195 consensus_peers.cc:485] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 -> Peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): Couldn't send request to peer 166d7170d98944e88de2eccda9d30545. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.888687 29305 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.888963 29306 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.889863 29304 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.890626 29308 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.893000 23864 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.895315 24392 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.897104 23590 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.899058 24522 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.901728 29306 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.902133 23994 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.903771 23731 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.906337 25941 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.916307 24392 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:45.918211 24522 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.918545 29374 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.919152 25941 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.919482 24260 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.919768 23731 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.919809 29374 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.920075 23864 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.920352 29107 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.920985 29374 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 14:10:45.921205 24327 consensus_peers.cc:485] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): Couldn't send request to peer 166d7170d98944e88de2eccda9d30545. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.921269 23994 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 14:10:45.921555 29374 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.923707 29107 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 14:10:45.925549 23994 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.925189 29375 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.927559 23581 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.928525 24522 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.929252 25941 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.931783 24260 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.931883 23731 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.935460 23864 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.941079 24392 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:45.942540 25941 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.942610 26071 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.942685 23731 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.942706 24260 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.942991 29374 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.943099 23864 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.943264 29107 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.943455 29375 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.943675 24522 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.943746 29375 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.944068 29375 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.944188 29374 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 14:10:45.945302 24327 consensus_peers.cc:485] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): Couldn't send request to peer 166d7170d98944e88de2eccda9d30545. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.945747 29377 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.946202 26071 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.947855 24522 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.947881 23864 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.948127 23581 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:45.951186 23731 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.951262 25941 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.952970 24260 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.960454 24392 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:45.962078 23864 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.962124 23731 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.962213 24522 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 14:10:45.962365 24327 consensus_peers.cc:485] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 -> Peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669): Couldn't send request to peer 166d7170d98944e88de2eccda9d30545. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:45.962570 29046 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.962790 25941 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.962812 26071 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.962894 29046 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.963019 29376 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.963639 29046 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.963774 29378 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.966120 29378 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.966367 24260 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:45.966506 26071 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.966795 29046 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:45.966810 23731 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.968192 23864 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.968398 24522 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.969146 23590 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.970592 24260 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:45.974215 25941 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 166d7170d98944e88de2eccda9d30545 (127.6.153.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } }
I20240501 14:10:46.203590 29464 ts_tablet_manager.cc:902] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:46.204135 29464 tablet_copy_client.cc:308] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:46.204635 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet 697668f5caf34e63a9e5953c180959ee from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:56835)
I20240501 14:10:46.204736 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet 697668f5caf34e63a9e5953c180959ee from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:56835: session id = a308c0ed71f44f57967c97239d1198f7-697668f5caf34e63a9e5953c180959ee
I20240501 14:10:46.205718 23752 tablet_copy_source_session.cc:205] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.206509 29464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697668f5caf34e63a9e5953c180959ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.208742 29464 tablet_copy_client.cc:783] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.208882 29464 tablet_copy_client.cc:647] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.210638 29464 tablet_copy_client.cc:523] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.211829 29464 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:46.235776 29464 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.236023 29464 tablet_bootstrap.cc:492] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:46.236161 29464 ts_tablet_manager.cc:1367] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.024s user 0.017s sys 0.006s
I20240501 14:10:46.236707 29464 raft_consensus.cc:348] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:46.236840 29464 raft_consensus.cc:729] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:46.236923 29464 consensus_queue.cc:260] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:46.237262 29464 ts_tablet_manager.cc:1398] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:46.237726 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-697668f5caf34e63a9e5953c180959ee received from {username='slave'} at 127.6.153.201:56835
I20240501 14:10:46.237802 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-697668f5caf34e63a9e5953c180959ee on tablet 697668f5caf34e63a9e5953c180959ee with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:46.286253 29500 ts_tablet_manager.cc:902] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Initiating tablet copy from peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:46.286996 29500 tablet_copy_client.cc:308] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:46669
I20240501 14:10:46.287545 24148 tablet_copy_service.cc:140] P 166d7170d98944e88de2eccda9d30545: Received BeginTabletCopySession request for tablet fee07797b5c84ad8aa71d10c1ac24fe1 from peer 8ebe0a098b0f4ed4861a5605717fecbd ({username='slave'} at 127.6.153.195:49731)
I20240501 14:10:46.287631 24148 tablet_copy_service.cc:161] P 166d7170d98944e88de2eccda9d30545: Beginning new tablet copy session on tablet fee07797b5c84ad8aa71d10c1ac24fe1 from peer 8ebe0a098b0f4ed4861a5605717fecbd at {username='slave'} at 127.6.153.195:49731: session id = 8ebe0a098b0f4ed4861a5605717fecbd-fee07797b5c84ad8aa71d10c1ac24fe1
I20240501 14:10:46.288394 24148 tablet_copy_source_session.cc:205] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.289335 29500 ts_tablet_manager.cc:1875] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:46.291065 29500 ts_tablet_manager.cc:1888] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240501 14:10:46.291222 29500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee07797b5c84ad8aa71d10c1ac24fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.292186 29500 tablet_copy_client.cc:783] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.292317 29500 tablet_copy_client.cc:647] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.293642 29500 tablet_copy_client.cc:523] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.294860 29500 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap starting.
I20240501 14:10:46.295358 29505 ts_tablet_manager.cc:902] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:46.296109 29505 tablet_copy_client.cc:308] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:46.296712 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet d25ef07189ac43749efc549299dc89f8 from peer 2502b9e8098b4bc5af7204835fce444b ({username='slave'} at 127.6.153.194:38315)
I20240501 14:10:46.296798 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet d25ef07189ac43749efc549299dc89f8 from peer 2502b9e8098b4bc5af7204835fce444b at {username='slave'} at 127.6.153.194:38315: session id = 2502b9e8098b4bc5af7204835fce444b-d25ef07189ac43749efc549299dc89f8
I20240501 14:10:46.297417 23752 tablet_copy_source_session.cc:205] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.298214 29505 ts_tablet_manager.cc:1875] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:46.300004 29505 ts_tablet_manager.cc:1888] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240501 14:10:46.300158 29505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.301198 29505 tablet_copy_client.cc:783] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.301344 29505 tablet_copy_client.cc:647] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.303287 29505 tablet_copy_client.cc:523] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.304594 29505 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap starting.
I20240501 14:10:46.324977 29516 ts_tablet_manager.cc:902] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:46.325721 29500 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.326035 29500 tablet_bootstrap.cc:492] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Bootstrap complete.
I20240501 14:10:46.326220 29500 ts_tablet_manager.cc:1367] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent bootstrapping tablet: real 0.031s user 0.025s sys 0.004s
I20240501 14:10:46.326629 29500 raft_consensus.cc:348] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } }
I20240501 14:10:46.326777 29500 raft_consensus.cc:729] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8ebe0a098b0f4ed4861a5605717fecbd, State: Initialized, Role: LEARNER
I20240501 14:10:46.325489 29516 tablet_copy_client.cc:308] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:38101
I20240501 14:10:46.327697 24280 tablet_copy_service.cc:140] P dbeb6a07fa7a44019ae00c169c414529: Received BeginTabletCopySession request for tablet 3377f84deb6249668991e5a3f6d63876 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:45777)
I20240501 14:10:46.327672 29500 consensus_queue.cc:260] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: NON_VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: true } }
I20240501 14:10:46.327796 24280 tablet_copy_service.cc:161] P dbeb6a07fa7a44019ae00c169c414529: Beginning new tablet copy session on tablet 3377f84deb6249668991e5a3f6d63876 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:45777: session id = c958143102d04279ac25aa56214d0d4b-3377f84deb6249668991e5a3f6d63876
I20240501 14:10:46.328053 29500 ts_tablet_manager.cc:1398] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 14:10:46.328701 24280 tablet_copy_source_session.cc:205] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.328987 24148 tablet_copy_service.cc:342] P 166d7170d98944e88de2eccda9d30545: Request end of tablet copy session 8ebe0a098b0f4ed4861a5605717fecbd-fee07797b5c84ad8aa71d10c1ac24fe1 received from {username='slave'} at 127.6.153.195:49731
I20240501 14:10:46.329051 24148 tablet_copy_service.cc:433] P 166d7170d98944e88de2eccda9d30545: ending tablet copy session 8ebe0a098b0f4ed4861a5605717fecbd-fee07797b5c84ad8aa71d10c1ac24fe1 on tablet fee07797b5c84ad8aa71d10c1ac24fe1 with peer 8ebe0a098b0f4ed4861a5605717fecbd
I20240501 14:10:46.329310 29505 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.329568 29505 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Bootstrap complete.
I20240501 14:10:46.329737 29505 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Time spent bootstrapping tablet: real 0.025s user 0.019s sys 0.003s
I20240501 14:10:46.329908 29516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3377f84deb6249668991e5a3f6d63876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.330140 29505 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } }
I20240501 14:10:46.330278 29505 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2502b9e8098b4bc5af7204835fce444b, State: Initialized, Role: LEARNER
I20240501 14:10:46.330386 29505 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } }
I20240501 14:10:46.330680 29505 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:46.331321 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session 2502b9e8098b4bc5af7204835fce444b-d25ef07189ac43749efc549299dc89f8 received from {username='slave'} at 127.6.153.194:38315
I20240501 14:10:46.331410 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session 2502b9e8098b4bc5af7204835fce444b-d25ef07189ac43749efc549299dc89f8 on tablet d25ef07189ac43749efc549299dc89f8 with peer 2502b9e8098b4bc5af7204835fce444b
I20240501 14:10:46.333675 29516 tablet_copy_client.cc:783] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.333856 29516 tablet_copy_client.cc:647] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.337453 29516 tablet_copy_client.cc:523] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.338469 29516 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:46.339115 29533 ts_tablet_manager.cc:902] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Initiating tablet copy from peer 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621)
I20240501 14:10:46.344594 29533 tablet_copy_client.cc:308] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.193:35621
I20240501 14:10:46.345420 23752 tablet_copy_service.cc:140] P 24d90b55560e45feb5fe4b9482317a4c: Received BeginTabletCopySession request for tablet d25ef07189ac43749efc549299dc89f8 from peer 166d7170d98944e88de2eccda9d30545 ({username='slave'} at 127.6.153.196:38607)
I20240501 14:10:46.345510 23752 tablet_copy_service.cc:161] P 24d90b55560e45feb5fe4b9482317a4c: Beginning new tablet copy session on tablet d25ef07189ac43749efc549299dc89f8 from peer 166d7170d98944e88de2eccda9d30545 at {username='slave'} at 127.6.153.196:38607: session id = 166d7170d98944e88de2eccda9d30545-d25ef07189ac43749efc549299dc89f8
I20240501 14:10:46.346405 23752 tablet_copy_source_session.cc:205] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.347383 29533 ts_tablet_manager.cc:1875] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:46.349323 29533 ts_tablet_manager.cc:1888] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 14:10:46.349464 29533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25ef07189ac43749efc549299dc89f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.350524 29533 tablet_copy_client.cc:783] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.350661 29533 tablet_copy_client.cc:647] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.352044 29548 ts_tablet_manager.cc:902] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:46.352159 29533 tablet_copy_client.cc:523] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.353507 29533 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:46.355948 29548 tablet_copy_client.cc:308] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:46669
I20240501 14:10:46.356326 24148 tablet_copy_service.cc:140] P 166d7170d98944e88de2eccda9d30545: Received BeginTabletCopySession request for tablet de25122a01e845d2b35e4a1023789d3e from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:39207)
I20240501 14:10:46.356413 24148 tablet_copy_service.cc:161] P 166d7170d98944e88de2eccda9d30545: Beginning new tablet copy session on tablet de25122a01e845d2b35e4a1023789d3e from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:39207: session id = c958143102d04279ac25aa56214d0d4b-de25122a01e845d2b35e4a1023789d3e
I20240501 14:10:46.357208 24148 tablet_copy_source_session.cc:205] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.357894 29548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.359654 29548 tablet_copy_client.cc:783] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.359778 29548 tablet_copy_client.cc:647] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.360663 29548 tablet_copy_client.cc:523] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.361485 29548 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:46.370536 29516 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.370788 29516 tablet_bootstrap.cc:492] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:46.370927 29516 ts_tablet_manager.cc:1367] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.006s
I20240501 14:10:46.371273 29516 raft_consensus.cc:348] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:46.371409 29516 raft_consensus.cc:729] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:46.371500 29516 consensus_queue.cc:260] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:46.371768 29516 ts_tablet_manager.cc:1398] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:46.372134 24280 tablet_copy_service.cc:342] P dbeb6a07fa7a44019ae00c169c414529: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-3377f84deb6249668991e5a3f6d63876 received from {username='slave'} at 127.6.153.202:45777
I20240501 14:10:46.372203 24280 tablet_copy_service.cc:433] P dbeb6a07fa7a44019ae00c169c414529: ending tablet copy session c958143102d04279ac25aa56214d0d4b-3377f84deb6249668991e5a3f6d63876 on tablet 3377f84deb6249668991e5a3f6d63876 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:46.374897 29556 ts_tablet_manager.cc:902] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:46.375398 29556 tablet_copy_client.cc:308] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:46.375857 24411 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet a50dc90c18174aa289c75056c4c461b0 from peer 166d7170d98944e88de2eccda9d30545 ({username='slave'} at 127.6.153.196:41397)
I20240501 14:10:46.375949 24411 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet a50dc90c18174aa289c75056c4c461b0 from peer 166d7170d98944e88de2eccda9d30545 at {username='slave'} at 127.6.153.196:41397: session id = 166d7170d98944e88de2eccda9d30545-a50dc90c18174aa289c75056c4c461b0
I20240501 14:10:46.376789 24411 tablet_copy_source_session.cc:205] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.377574 29556 ts_tablet_manager.cc:1875] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:46.379568 29556 ts_tablet_manager.cc:1888] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 14:10:46.379776 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50dc90c18174aa289c75056c4c461b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.380601 29556 tablet_copy_client.cc:783] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.380904 29556 tablet_copy_client.cc:647] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.381652 29561 ts_tablet_manager.cc:902] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:46.382314 29556 tablet_copy_client.cc:523] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.382352 29561 tablet_copy_client.cc:308] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:46.382695 24411 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet 84da4677d89b42fe8c2418cd156c75ae from peer 166d7170d98944e88de2eccda9d30545 ({username='slave'} at 127.6.153.196:41397)
I20240501 14:10:46.382771 24411 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet 84da4677d89b42fe8c2418cd156c75ae from peer 166d7170d98944e88de2eccda9d30545 at {username='slave'} at 127.6.153.196:41397: session id = 166d7170d98944e88de2eccda9d30545-84da4677d89b42fe8c2418cd156c75ae
I20240501 14:10:46.383527 29556 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:46.383626 24411 tablet_copy_source_session.cc:205] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.384240 29561 ts_tablet_manager.cc:1875] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:46.386240 29533 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.386520 29533 tablet_bootstrap.cc:492] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Bootstrap complete.
I20240501 14:10:46.386659 29533 ts_tablet_manager.cc:1367] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.033s user 0.030s sys 0.000s
I20240501 14:10:46.386991 29533 raft_consensus.cc:348] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } }
I20240501 14:10:46.387118 29533 raft_consensus.cc:729] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: LEARNER
I20240501 14:10:46.387209 29533 consensus_queue.cc:260] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: NON_VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: true } }
I20240501 14:10:46.387621 29533 ts_tablet_manager.cc:1398] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:46.388095 23752 tablet_copy_service.cc:342] P 24d90b55560e45feb5fe4b9482317a4c: Request end of tablet copy session 166d7170d98944e88de2eccda9d30545-d25ef07189ac43749efc549299dc89f8 received from {username='slave'} at 127.6.153.196:38607
I20240501 14:10:46.388168 23752 tablet_copy_service.cc:433] P 24d90b55560e45feb5fe4b9482317a4c: ending tablet copy session 166d7170d98944e88de2eccda9d30545-d25ef07189ac43749efc549299dc89f8 on tablet d25ef07189ac43749efc549299dc89f8 with peer 166d7170d98944e88de2eccda9d30545
I20240501 14:10:46.390528 29561 ts_tablet_manager.cc:1888] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 14:10:46.391085 29561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84da4677d89b42fe8c2418cd156c75ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.392168 29561 tablet_copy_client.cc:783] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.392292 29561 tablet_copy_client.cc:647] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.393131 29561 tablet_copy_client.cc:523] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.394091 29561 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:46.398751 29568 ts_tablet_manager.cc:902] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Initiating tablet copy from peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:46.399499 29568 tablet_copy_client.cc:308] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:46669
I20240501 14:10:46.399972 24148 tablet_copy_service.cc:140] P 166d7170d98944e88de2eccda9d30545: Received BeginTabletCopySession request for tablet de25122a01e845d2b35e4a1023789d3e from peer dbeb6a07fa7a44019ae00c169c414529 ({username='slave'} at 127.6.153.197:60527)
I20240501 14:10:46.400063 24148 tablet_copy_service.cc:161] P 166d7170d98944e88de2eccda9d30545: Beginning new tablet copy session on tablet de25122a01e845d2b35e4a1023789d3e from peer dbeb6a07fa7a44019ae00c169c414529 at {username='slave'} at 127.6.153.197:60527: session id = dbeb6a07fa7a44019ae00c169c414529-de25122a01e845d2b35e4a1023789d3e
I20240501 14:10:46.400817 24148 tablet_copy_source_session.cc:205] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.401808 29568 ts_tablet_manager.cc:1875] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:46.403820 29568 ts_tablet_manager.cc:1888] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 14:10:46.404057 29568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.405503 29568 tablet_copy_client.cc:783] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.405717 29568 tablet_copy_client.cc:647] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.406895 29568 tablet_copy_client.cc:523] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.408497 29568 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Bootstrap starting.
I20240501 14:10:46.415755 29548 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.415999 29548 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:46.416147 29548 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.055s user 0.025s sys 0.003s
I20240501 14:10:46.416560 29548 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:46.416695 29548 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:46.416806 29548 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:46.419163 29533 ts_tablet_manager.cc:902] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Initiating tablet copy from peer 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897)
I20240501 14:10:46.419589 29533 tablet_copy_client.cc:308] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.198:41897
I20240501 14:10:46.419976 24411 tablet_copy_service.cc:140] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Received BeginTabletCopySession request for tablet 76fbefe89b8344e1a09101d404f32d9e from peer 166d7170d98944e88de2eccda9d30545 ({username='slave'} at 127.6.153.196:41397)
I20240501 14:10:46.420068 24411 tablet_copy_service.cc:161] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Beginning new tablet copy session on tablet 76fbefe89b8344e1a09101d404f32d9e from peer 166d7170d98944e88de2eccda9d30545 at {username='slave'} at 127.6.153.196:41397: session id = 166d7170d98944e88de2eccda9d30545-76fbefe89b8344e1a09101d404f32d9e
I20240501 14:10:46.420931 24411 tablet_copy_source_session.cc:205] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.421561 29533 ts_tablet_manager.cc:1875] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:46.422986 29533 ts_tablet_manager.cc:1888] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 14:10:46.423118 29533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76fbefe89b8344e1a09101d404f32d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.424046 29533 tablet_copy_client.cc:783] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.424171 29533 tablet_copy_client.cc:647] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.425230 29533 tablet_copy_client.cc:523] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.426272 29533 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:46.426298 29548 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20240501 14:10:46.426690 24148 tablet_copy_service.cc:342] P 166d7170d98944e88de2eccda9d30545: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-de25122a01e845d2b35e4a1023789d3e received from {username='slave'} at 127.6.153.202:39207
I20240501 14:10:46.426771 24148 tablet_copy_service.cc:433] P 166d7170d98944e88de2eccda9d30545: ending tablet copy session c958143102d04279ac25aa56214d0d4b-de25122a01e845d2b35e4a1023789d3e on tablet de25122a01e845d2b35e4a1023789d3e with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:46.430908 29556 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.431166 29556 tablet_bootstrap.cc:492] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Bootstrap complete.
I20240501 14:10:46.431322 29556 ts_tablet_manager.cc:1367] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.048s user 0.026s sys 0.000s
I20240501 14:10:46.431713 29556 raft_consensus.cc:348] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:46.431833 29556 raft_consensus.cc:729] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: LEARNER
I20240501 14:10:46.431926 29556 consensus_queue.cc:260] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:46.432626 29556 ts_tablet_manager.cc:1398] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:46.433008 24411 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session 166d7170d98944e88de2eccda9d30545-a50dc90c18174aa289c75056c4c461b0 received from {username='slave'} at 127.6.153.196:41397
I20240501 14:10:46.433075 24411 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session 166d7170d98944e88de2eccda9d30545-a50dc90c18174aa289c75056c4c461b0 on tablet a50dc90c18174aa289c75056c4c461b0 with peer 166d7170d98944e88de2eccda9d30545
I20240501 14:10:46.433492 29561 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.433715 29561 tablet_bootstrap.cc:492] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Bootstrap complete.
I20240501 14:10:46.433961 29561 ts_tablet_manager.cc:1367] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.040s user 0.029s sys 0.000s
I20240501 14:10:46.434903 29561 raft_consensus.cc:348] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:46.435000 29561 raft_consensus.cc:729] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: LEARNER
I20240501 14:10:46.435084 29561 consensus_queue.cc:260] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:46.435456 29561 ts_tablet_manager.cc:1398] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 14:10:46.435822 24411 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session 166d7170d98944e88de2eccda9d30545-84da4677d89b42fe8c2418cd156c75ae received from {username='slave'} at 127.6.153.196:41397
I20240501 14:10:46.435892 24411 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session 166d7170d98944e88de2eccda9d30545-84da4677d89b42fe8c2418cd156c75ae on tablet 84da4677d89b42fe8c2418cd156c75ae with peer 166d7170d98944e88de2eccda9d30545
I20240501 14:10:46.440060 29568 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.440317 29568 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Bootstrap complete.
I20240501 14:10:46.440467 29568 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Time spent bootstrapping tablet: real 0.032s user 0.022s sys 0.003s
I20240501 14:10:46.440922 29568 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:46.441053 29568 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbeb6a07fa7a44019ae00c169c414529, State: Initialized, Role: LEARNER
I20240501 14:10:46.441146 29568 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: NON_VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:46.441438 29568 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 14:10:46.441962 24148 tablet_copy_service.cc:342] P 166d7170d98944e88de2eccda9d30545: Request end of tablet copy session dbeb6a07fa7a44019ae00c169c414529-de25122a01e845d2b35e4a1023789d3e received from {username='slave'} at 127.6.153.197:60527
I20240501 14:10:46.442023 24148 tablet_copy_service.cc:433] P 166d7170d98944e88de2eccda9d30545: ending tablet copy session dbeb6a07fa7a44019ae00c169c414529-de25122a01e845d2b35e4a1023789d3e on tablet de25122a01e845d2b35e4a1023789d3e with peer dbeb6a07fa7a44019ae00c169c414529
I20240501 14:10:46.456228 29533 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.456480 29533 tablet_bootstrap.cc:492] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Bootstrap complete.
I20240501 14:10:46.456616 29533 ts_tablet_manager.cc:1367] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.030s user 0.014s sys 0.007s
I20240501 14:10:46.456985 29533 raft_consensus.cc:348] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:46.457082 29533 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: LEARNER
I20240501 14:10:46.457165 29533 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:46.457433 29533 ts_tablet_manager.cc:1398] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:46.457782 24411 tablet_copy_service.cc:342] P 7e6e9293a4614dabb5ad89fe3f4d5d04: Request end of tablet copy session 166d7170d98944e88de2eccda9d30545-76fbefe89b8344e1a09101d404f32d9e received from {username='slave'} at 127.6.153.196:41397
I20240501 14:10:46.457849 24411 tablet_copy_service.cc:433] P 7e6e9293a4614dabb5ad89fe3f4d5d04: ending tablet copy session 166d7170d98944e88de2eccda9d30545-76fbefe89b8344e1a09101d404f32d9e on tablet 76fbefe89b8344e1a09101d404f32d9e with peer 166d7170d98944e88de2eccda9d30545
I20240501 14:10:46.486690 29533 ts_tablet_manager.cc:902] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Initiating tablet copy from peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:46.487084 29533 tablet_copy_client.cc:308] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:38101
I20240501 14:10:46.487460 24280 tablet_copy_service.cc:140] P dbeb6a07fa7a44019ae00c169c414529: Received BeginTabletCopySession request for tablet 34a278486c8b447d8dc4c3241743de60 from peer 166d7170d98944e88de2eccda9d30545 ({username='slave'} at 127.6.153.196:32915)
I20240501 14:10:46.487551 24280 tablet_copy_service.cc:161] P dbeb6a07fa7a44019ae00c169c414529: Beginning new tablet copy session on tablet 34a278486c8b447d8dc4c3241743de60 from peer 166d7170d98944e88de2eccda9d30545 at {username='slave'} at 127.6.153.196:32915: session id = 166d7170d98944e88de2eccda9d30545-34a278486c8b447d8dc4c3241743de60
I20240501 14:10:46.488305 24280 tablet_copy_source_session.cc:205] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:46.488876 29533 ts_tablet_manager.cc:1875] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:46.490492 29533 ts_tablet_manager.cc:1888] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 14:10:46.490613 29533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:46.491395 29533 tablet_copy_client.cc:783] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:46.491503 29533 tablet_copy_client.cc:647] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:46.492389 29533 tablet_copy_client.cc:523] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:46.493279 29533 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Bootstrap starting.
I20240501 14:10:46.510959 29533 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:46.511149 29533 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Bootstrap complete.
I20240501 14:10:46.511286 29533 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20240501 14:10:46.511610 29533 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:46.511708 29533 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 166d7170d98944e88de2eccda9d30545, State: Initialized, Role: LEARNER
I20240501 14:10:46.511795 29533 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } }
I20240501 14:10:46.512025 29533 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:46.512399 24280 tablet_copy_service.cc:342] P dbeb6a07fa7a44019ae00c169c414529: Request end of tablet copy session 166d7170d98944e88de2eccda9d30545-34a278486c8b447d8dc4c3241743de60 received from {username='slave'} at 127.6.153.196:32915
I20240501 14:10:46.512483 24280 tablet_copy_service.cc:433] P dbeb6a07fa7a44019ae00c169c414529: ending tablet copy session 166d7170d98944e88de2eccda9d30545-34a278486c8b447d8dc4c3241743de60 on tablet 34a278486c8b447d8dc4c3241743de60 with peer 166d7170d98944e88de2eccda9d30545
I20240501 14:10:46.617203 25808 raft_consensus.cc:1203] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:46.696323 23864 raft_consensus.cc:1203] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 14:10:46.797401 25941 raft_consensus.cc:1203] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 14:10:46.837718 24260 raft_consensus.cc:1203] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.8->[1.9-1.9] Dedup: 1.9->[]
I20240501 14:10:46.882639 23994 raft_consensus.cc:1203] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 14:10:46.903649 24126 raft_consensus.cc:1203] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 14:10:46.910509 24126 raft_consensus.cc:1203] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:46.925824 24126 raft_consensus.cc:1203] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 14:10:46.937229 25941 raft_consensus.cc:1203] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.8->[1.9-1.9] Dedup: 1.9->[]
I20240501 14:10:46.941562 24126 raft_consensus.cc:1203] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 14:10:47.063987 24126 raft_consensus.cc:1203] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 14:10:47.218284 29332 raft_consensus.cc:1050] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: attempting to promote NON_VOTER dbeb6a07fa7a44019ae00c169c414529 to VOTER
I20240501 14:10:47.218765 29332 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:47.220606 24522 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.220639 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.220672 23864 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.220734 25941 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.221026 29332 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.221066 25808 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.221187 29795 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.221530 29332 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.221909 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.222311 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEARNER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.222612 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.221686 29796 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.223636 29332 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.224257 29797 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.224623 29795 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.224967 29265 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.226100 29797 raft_consensus.cc:1013] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: attempt to promote peer c958143102d04279ac25aa56214d0d4b: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240501 14:10:47.227003 29797 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.227433 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.229575 23994 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.229743 25941 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.230207 25793 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.232167 23731 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.232177 23587 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.233278 23864 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.233922 29332 raft_consensus.cc:1050] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:47.233996 29259 raft_consensus.cc:1050] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:47.234357 29332 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:47.234489 29259 consensus_queue.cc:237] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:47.235024 24522 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.236649 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:47.236972 25941 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:47.237036 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:47.237097 29797 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:47.237740 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:47.238075 29798 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:47.238368 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:47.238317 29798 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:47.238943 25808 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:47.239284 29253 raft_consensus.cc:1050] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER 2502b9e8098b4bc5af7204835fce444b to VOTER
I20240501 14:10:47.239573 25941 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.239575 24521 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:47.239768 25808 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.239786 29253 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } }
I20240501 14:10:47.239895 23862 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:47.239914 24392 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.240193 29796 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.240295 29256 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.240460 29796 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:47.240576 29256 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.241443 29796 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:47.241302 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:47.241690 29796 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:47.241818 29266 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:47.243185 29820 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.243590 24524 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.244057 24126 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.244354 24522 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.244525 25793 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.244551 26071 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.244743 29819 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.245194 29798 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.246599 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.246857 29256 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.247246 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.247285 23994 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.247383 24390 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.248816 23862 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.250176 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.250454 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.250883 24524 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.251947 29253 raft_consensus.cc:1013] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: attempt to promote peer 166d7170d98944e88de2eccda9d30545: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240501 14:10:47.251969 24260 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.252347 23862 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.252633 23994 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.253386 24391 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.253453 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.253680 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.253907 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.254101 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.255546 23993 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.255898 29259 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.256068 24126 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.257072 29259 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.257148 23864 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.257467 29259 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.259503 25940 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.261267 25808 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.261731 23731 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.261763 29253 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.262063 24521 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.263870 24392 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.263931 23862 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.265949 23994 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.266217 24126 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.267572 29819 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.268108 29820 raft_consensus.cc:1050] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c: attempting to promote NON_VOTER 166d7170d98944e88de2eccda9d30545 to VOTER
I20240501 14:10:47.268570 29820 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } }
I20240501 14:10:47.268522 24260 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.269306 25940 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.270956 24260 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:47.271426 23862 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:47.271600 24521 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.272186 29846 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:47.273283 23993 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:47.273159 23994 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.273913 23864 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.274183 24126 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.275372 24127 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:47.275815 29846 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:47.275944 29259 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:47.277945 24521 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:47.277957 29830 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:47.278342 29253 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:47.281251 23581 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 10 to 11, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.281550 23581 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.281810 25793 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:47.282174 29259 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.282092 25798 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.282373 26071 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:47.282691 29259 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.283450 26071 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.284022 25793 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.284324 29259 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.284682 24260 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.285136 24391 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:47.286767 24521 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.287029 23862 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.287184 24127 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.289464 24392 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.290273 23589 catalog_manager.cc:5484] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 8 to 9, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.291934 29253 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:47.292925 25793 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.293061 23994 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.293155 26071 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.296308 24392 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.297801 24127 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:47.298174 23590 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 13 to 14, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { replace: true } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.300608 25940 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.300938 23731 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.301333 29815 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.301627 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.302160 25793 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.302690 24391 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.303090 29266 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.303156 23862 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.303403 29266 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.303524 29797 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.304575 23994 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.305346 29797 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.305483 29266 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.305647 29798 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.307034 29046 raft_consensus.cc:1050] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER 166d7170d98944e88de2eccda9d30545 to VOTER
I20240501 14:10:47.307557 29046 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.309264 29797 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.310618 25941 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.311364 29374 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.311703 23730 consensus_queue.cc:237] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:47.311934 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet de25122a01e845d2b35e4a1023789d3e with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.312112 23731 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.312870 23729 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } }
I20240501 14:10:47.313074 24259 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.312947 23864 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.313583 25939 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.314786 29696 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.311745 23732 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.315177 29696 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.316028 24259 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:47.316392 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:47.316483 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.317055 24522 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.317754 26071 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.317806 24524 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:47.318137 29374 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.318176 29256 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.318336 24128 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:47.318826 23583 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.319499 29259 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:47.319588 24126 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.320211 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:47.320477 29256 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.321010 25798 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.321482 23863 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:47.321645 26072 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:47.322026 23864 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.322026 23993 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.322573 29256 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.322876 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:47.322991 29253 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.323184 24391 raft_consensus.cc:1261] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.323204 24127 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.323465 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.323623 23862 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.324008 29046 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.324858 29374 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.325208 24521 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.325299 29253 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.325508 29374 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.325565 29253 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:47.325609 24391 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:47.325803 29253 consensus_queue.cc:1035] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.326010 29256 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.327373 23993 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.327926 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d25ef07189ac43749efc549299dc89f8 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.328284 23864 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.328445 24126 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.328827 25798 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:47.329157 24522 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.329331 29253 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:47.330279 29256 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:47.330822 24260 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.331905 25808 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.331969 24391 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.332760 23587 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.334436 29259 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.335117 23863 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.336431 24524 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.336553 24126 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.337231 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 697668f5caf34e63a9e5953c180959ee with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.337777 23994 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.338922 25793 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false replace: true } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.339406 26072 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.340140 24504 tablet_service.cc:1514] Processing DeleteTablet for tablet de25122a01e845d2b35e4a1023789d3e with delete_type TABLET_DATA_TOMBSTONED (TS 40b69f74de5a447f86c1445f8e84850d not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:58450
I20240501 14:10:47.342432 24392 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.342986 29752 raft_consensus.cc:1050] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:47.343477 29752 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:47.343655 29046 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.345847 26072 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.346426 23729 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.347239 23732 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.347716 23976 tablet_service.cc:1514] Processing DeleteTablet for tablet d25ef07189ac43749efc549299dc89f8 with delete_type TABLET_DATA_TOMBSTONED (TS 8ebe0a098b0f4ed4861a5605717fecbd not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:58422
I20240501 14:10:47.347787 26072 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.348173 24392 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.348345 25941 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.348595 25793 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.348742 29690 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.348811 23589 catalog_manager.cc:5484] T 697668f5caf34e63a9e5953c180959ee P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 9 to 10, VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.342218 25808 raft_consensus.cc:2932] T 697668f5caf34e63a9e5953c180959ee P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:47.349037 24521 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.350152 29885 tablet_replica.cc:330] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: stopping tablet replica
I20240501 14:10:47.350812 29308 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.351125 29308 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.351444 29308 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.351624 29885 raft_consensus.cc:2229] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.351586 24127 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.351722 29308 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.351962 29885 raft_consensus.cc:2258] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.352227 29876 tablet_replica.cc:330] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: stopping tablet replica
I20240501 14:10:47.354429 24257 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.354614 23993 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.354702 23862 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.354940 29690 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.355378 29885 ts_tablet_manager.cc:1875] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.354529 24521 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.357010 25940 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.357309 29876 raft_consensus.cc:2229] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.357522 29752 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.357568 29876 raft_consensus.cc:2258] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.359526 24127 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:47.359608 29885 ts_tablet_manager.cc:1888] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.14
I20240501 14:10:47.359671 29885 log.cc:1196] T d25ef07189ac43749efc549299dc89f8 P 8ebe0a098b0f4ed4861a5605717fecbd: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-2/wal/wals/d25ef07189ac43749efc549299dc89f8
I20240501 14:10:47.359845 23581 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.360098 29876 ts_tablet_manager.cc:1875] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.360695 23577 catalog_manager.cc:4830] TS 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313): tablet d25ef07189ac43749efc549299dc89f8 (table rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]) successfully deleted
I20240501 14:10:47.361102 23732 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.361480 29815 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.363323 25941 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.363683 29815 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.364495 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.364549 23993 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.364945 29266 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.365089 23862 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.365281 29266 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.366462 24257 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.366467 29266 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.366803 29814 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.367075 29876 ts_tablet_manager.cc:1888] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.11
I20240501 14:10:47.367136 29876 log.cc:1196] T de25122a01e845d2b35e4a1023789d3e P 40b69f74de5a447f86c1445f8e84850d: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/wals/de25122a01e845d2b35e4a1023789d3e
I20240501 14:10:47.367919 23576 catalog_manager.cc:4830] TS 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305): tablet de25122a01e845d2b35e4a1023789d3e (table rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]) successfully deleted
I20240501 14:10:47.368927 29814 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.369299 29883 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.370249 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.371851 23732 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.372699 23993 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.373266 23587 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 12 to 13, VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) evicted. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:47.373898 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet de25122a01e845d2b35e4a1023789d3e with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.374155 25941 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.375962 25808 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.376225 25941 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.377722 23863 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.378379 24392 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.380388 23588 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.381471 23993 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.383692 25921 tablet_service.cc:1514] Processing DeleteTablet for tablet 697668f5caf34e63a9e5953c180959ee with delete_type TABLET_DATA_TOMBSTONED (TS c958143102d04279ac25aa56214d0d4b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:54086
I20240501 14:10:47.384910 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.386337 26072 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.388444 24392 consensus_queue.cc:237] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.389715 23732 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.390337 23729 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.390462 24521 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c958143102d04279ac25aa56214d0d4b (127.6.153.202) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false replace: true } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.391772 29860 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.392340 24126 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.392551 24257 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.393514 23863 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.393577 29862 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.393991 29862 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.394575 29861 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.394659 29860 raft_consensus.cc:1050] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER 166d7170d98944e88de2eccda9d30545 to VOTER
I20240501 14:10:47.395134 29860 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.398643 24257 consensus_queue.cc:237] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:47.400218 23993 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.400550 23732 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.400748 25941 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.401001 29882 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.401212 25940 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.401296 25808 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.401858 29859 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.402021 29879 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.401912 24392 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.402606 24521 raft_consensus.cc:1261] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.402988 26072 raft_consensus.cc:1261] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.403210 29882 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.403308 29046 consensus_queue.cc:1035] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.403712 29752 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.403668 29046 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.404065 29752 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.404724 23863 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.405130 23732 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.405393 24126 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.405794 29883 consensus_queue.cc:1035] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.405970 24524 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.406782 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a50dc90c18174aa289c75056c4c461b0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.406977 29859 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.407105 23863 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.407541 29862 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.407480 29882 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.408254 29859 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.408521 24260 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.408553 29859 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.408609 23586 catalog_manager.cc:5484] T a50dc90c18174aa289c75056c4c461b0 P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:47.409245 24126 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.409488 23993 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.409652 23730 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.411767 25941 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.412637 26071 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.412940 29862 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.412823 23729 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.413237 25940 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.413434 24504 tablet_service.cc:1514] Processing DeleteTablet for tablet a50dc90c18174aa289c75056c4c461b0 with delete_type TABLET_DATA_TOMBSTONED (TS 40b69f74de5a447f86c1445f8e84850d not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:58450
I20240501 14:10:47.414144 29876 tablet_replica.cc:330] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: stopping tablet replica
I20240501 14:10:47.414263 29876 raft_consensus.cc:2229] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.414507 29876 raft_consensus.cc:2258] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.414865 29901 tablet_replica.cc:330] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: stopping tablet replica
I20240501 14:10:47.415114 25772 tablet_service.cc:1514] Processing DeleteTablet for tablet de25122a01e845d2b35e4a1023789d3e with delete_type TABLET_DATA_TOMBSTONED (TS a308c0ed71f44f57967c97239d1198f7 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:42134
I20240501 14:10:47.415164 29901 raft_consensus.cc:2229] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.415037 23583 catalog_manager.cc:5484] T 3377f84deb6249668991e5a3f6d63876 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 12 to 13, VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) evicted. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.415529 29901 raft_consensus.cc:2258] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.416775 29862 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.417374 29862 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.417300 25940 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.417933 29914 tablet_replica.cc:330] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: stopping tablet replica
I20240501 14:10:47.418324 26072 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.418671 29901 ts_tablet_manager.cc:1875] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.418885 29914 raft_consensus.cc:2229] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.419247 29914 raft_consensus.cc:2258] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.419898 29876 ts_tablet_manager.cc:1875] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.420333 29914 ts_tablet_manager.cc:1875] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.421878 29876 ts_tablet_manager.cc:1888] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 14:10:47.421938 29876 log.cc:1196] T a50dc90c18174aa289c75056c4c461b0 P 40b69f74de5a447f86c1445f8e84850d: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/wals/a50dc90c18174aa289c75056c4c461b0
I20240501 14:10:47.422106 24524 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.422346 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3377f84deb6249668991e5a3f6d63876 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.422454 29901 ts_tablet_manager.cc:1888] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:47.422529 29901 log.cc:1196] T 697668f5caf34e63a9e5953c180959ee P c958143102d04279ac25aa56214d0d4b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-9/wal/wals/697668f5caf34e63a9e5953c180959ee
I20240501 14:10:47.422987 23574 catalog_manager.cc:4830] TS c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): tablet 697668f5caf34e63a9e5953c180959ee (table rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]) successfully deleted
I20240501 14:10:47.423712 24257 raft_consensus.cc:2932] T a50dc90c18174aa289c75056c4c461b0 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.424182 25808 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.424932 23576 catalog_manager.cc:4830] TS 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305): tablet a50dc90c18174aa289c75056c4c461b0 (table rebalance_test_table_5 [id=26187cf9b136457b8fe769a19cd27e32]) successfully deleted
I20240501 14:10:47.426806 24392 raft_consensus.cc:2932] T 3377f84deb6249668991e5a3f6d63876 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:47.433758 29860 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.428192 29914 ts_tablet_manager.cc:1888] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 14:10:47.435359 29914 log.cc:1196] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-8/wal/wals/de25122a01e845d2b35e4a1023789d3e
I20240501 14:10:47.436359 26052 tablet_service.cc:1514] Processing DeleteTablet for tablet 3377f84deb6249668991e5a3f6d63876 with delete_type TABLET_DATA_TOMBSTONED (TS 6d832f3f723641778e60b3fd855ff98e not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:36118
I20240501 14:10:47.436326 24126 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.436798 23732 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.436913 29923 tablet_replica.cc:330] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: stopping tablet replica
I20240501 14:10:47.436916 26071 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.437228 29923 raft_consensus.cc:2229] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.437508 29923 raft_consensus.cc:2258] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.439385 23574 catalog_manager.cc:4830] TS a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): tablet de25122a01e845d2b35e4a1023789d3e (table rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]) successfully deleted
I20240501 14:10:47.439273 23586 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.440472 29923 ts_tablet_manager.cc:1875] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.440481 25941 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.443053 29923 ts_tablet_manager.cc:1888] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 14:10:47.443173 29923 log.cc:1196] T 3377f84deb6249668991e5a3f6d63876 P 6d832f3f723641778e60b3fd855ff98e: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-7/wal/wals/3377f84deb6249668991e5a3f6d63876
I20240501 14:10:47.443075 23863 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.445914 24257 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.447152 23574 catalog_manager.cc:4830] TS 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): tablet 3377f84deb6249668991e5a3f6d63876 (table rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]) successfully deleted
I20240501 14:10:47.449193 24392 consensus_queue.cc:237] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.450722 23732 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.451670 24126 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.451843 26071 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.452104 25941 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.452116 23863 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.452533 29860 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.452762 29864 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.453069 29860 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.453624 24257 raft_consensus.cc:1261] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.454082 29860 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.454262 29864 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.455753 29864 consensus_queue.cc:1035] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.455976 29815 raft_consensus.cc:1050] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545: attempting to promote NON_VOTER 8ebe0a098b0f4ed4861a5605717fecbd to VOTER
I20240501 14:10:47.456420 29815 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } }
I20240501 14:10:47.456548 29863 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.457717 24257 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.458246 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 84da4677d89b42fe8c2418cd156c75ae with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.458729 26071 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.459460 24522 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.459486 23729 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.459843 29814 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.459904 23582 catalog_manager.cc:5484] T 84da4677d89b42fe8c2418cd156c75ae P 7e6e9293a4614dabb5ad89fe3f4d5d04 reported cstate change: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:47.460147 29798 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.460780 23993 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEARNER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.461076 23862 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.461321 29798 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.463153 24504 tablet_service.cc:1514] Processing DeleteTablet for tablet 84da4677d89b42fe8c2418cd156c75ae with delete_type TABLET_DATA_TOMBSTONED (TS 40b69f74de5a447f86c1445f8e84850d not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:58450
I20240501 14:10:47.463357 29876 tablet_replica.cc:330] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: stopping tablet replica
I20240501 14:10:47.463472 29876 raft_consensus.cc:2229] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.463477 23863 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.463689 29876 pending_rounds.cc:62] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: Trying to abort 1 pending ops.
I20240501 14:10:47.463757 29876 pending_rounds.cc:68] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: Aborting op as it isn't in flight: id { term: 1 index: 12 } timestamp: 7022889563729002496 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "84da4677d89b42fe8c2418cd156c75ae" old_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: NON_VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: true } } } new_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } } }
I20240501 14:10:47.463915 29876 raft_consensus.cc:2894] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Aborting config change with OpId 1.12: Aborted: Op aborted
I20240501 14:10:47.463985 29876 raft_consensus.cc:2866] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240501 14:10:47.464033 29876 raft_consensus.cc:2258] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.464260 29798 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.464399 24126 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.465261 25940 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.465590 25808 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.466147 29798 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.466692 23732 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.467002 29814 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.468116 25941 raft_consensus.cc:2932] T 84da4677d89b42fe8c2418cd156c75ae P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.470297 24257 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:47.471504 29814 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:47.473387 29876 ts_tablet_manager.cc:1875] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.474128 29798 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.474512 24257 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.475499 29876 ts_tablet_manager.cc:1888] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 14:10:47.475570 29876 log.cc:1196] T 84da4677d89b42fe8c2418cd156c75ae P 40b69f74de5a447f86c1445f8e84850d: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/wals/84da4677d89b42fe8c2418cd156c75ae
I20240501 14:10:47.476106 23576 catalog_manager.cc:4830] TS 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305): tablet 84da4677d89b42fe8c2418cd156c75ae (table rebalance_test_table_4 [id=dbd9141619554c85b59835b716c7ee31]) successfully deleted
I20240501 14:10:47.477077 23589 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:47.477260 24522 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.478119 23863 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.480918 25808 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.482066 23993 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.483753 23732 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.484388 25940 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.489089 24126 consensus_queue.cc:237] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } }
I20240501 14:10:47.490815 23993 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.490902 24522 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.490998 23732 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.491127 25940 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.491151 29796 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.491289 29814 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.491518 25808 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.491591 29795 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.492244 24257 raft_consensus.cc:1261] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:47.492354 29798 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.492630 29796 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.493916 29798 consensus_queue.cc:1035] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:47.495476 29795 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.499903 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fee07797b5c84ad8aa71d10c1ac24fe1 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.500716 23993 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.501580 25808 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.501806 24257 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.503943 25940 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.504088 23732 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.505864 29883 raft_consensus.cc:1050] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: attempting to promote NON_VOTER 166d7170d98944e88de2eccda9d30545 to VOTER
I20240501 14:10:47.506297 23586 catalog_manager.cc:5484] T fee07797b5c84ad8aa71d10c1ac24fe1 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 12 to 13, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.506383 29883 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.507119 24522 raft_consensus.cc:2932] T fee07797b5c84ad8aa71d10c1ac24fe1 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 2502b9e8098b4bc5af7204835fce444b (127.6.153.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } attrs { promote: false } } }
I20240501 14:10:47.508790 24392 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.508994 23993 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.509320 23863 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.509616 29882 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.510541 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet fee07797b5c84ad8aa71d10c1ac24fe1 with delete_type TABLET_DATA_TOMBSTONED (TS 2502b9e8098b4bc5af7204835fce444b not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:33618
I20240501 14:10:47.510915 29953 tablet_replica.cc:330] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: stopping tablet replica
I20240501 14:10:47.510996 29308 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.511132 29953 raft_consensus.cc:2229] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.511194 24126 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.511308 29308 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.511420 29953 raft_consensus.cc:2258] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.511677 29308 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.512259 29863 raft_consensus.cc:1050] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: attempting to promote NON_VOTER 166d7170d98944e88de2eccda9d30545 to VOTER
I20240501 14:10:47.512773 29863 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.513052 29953 ts_tablet_manager.cc:1875] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.514765 23994 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.515146 29374 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.515419 24524 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.515676 23729 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.515695 29953 ts_tablet_manager.cc:1888] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 14:10:47.515942 29953 log.cc:1196] T fee07797b5c84ad8aa71d10c1ac24fe1 P 2502b9e8098b4bc5af7204835fce444b: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-1/wal/wals/fee07797b5c84ad8aa71d10c1ac24fe1
I20240501 14:10:47.516083 23862 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.516050 29046 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.516356 29046 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.516685 25941 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.517057 24257 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.517057 24128 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 LEARNER]: Refusing update from remote peer 7e6e9293a4614dabb5ad89fe3f4d5d04: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.517120 29374 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.517436 29374 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.517599 25940 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.518082 23577 catalog_manager.cc:4830] TS 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427): tablet fee07797b5c84ad8aa71d10c1ac24fe1 (table rebalance_test_table_2 [id=51626c2fe4774fbe87400f7d1405371e]) successfully deleted
I20240501 14:10:47.518544 29859 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.518842 29046 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.519174 29882 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.519861 23732 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.520529 24522 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 14:10:47.520857 29882 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.522755 29308 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 14:10:47.525487 29859 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.525970 24524 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.528180 23862 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.528836 29308 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.529891 23582 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d reported cstate change: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.530812 24392 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.530807 25941 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.531993 23729 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.532456 24260 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.533103 24128 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.535287 24128 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.535758 23582 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.536590 24522 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.537410 23862 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.541554 25940 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.541554 23993 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.547886 23732 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.549396 24257 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.551400 23862 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.551487 24522 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.551728 23732 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.551784 29690 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.552028 25940 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.552418 29690 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.552567 29881 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.552680 29690 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.553145 24128 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.553136 23994 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 166d7170d98944e88de2eccda9d30545 (127.6.153.196) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.554539 23993 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 14:10:47.556201 29880 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.557178 29690 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:47.557698 29880 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.559491 25940 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.559633 23862 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.561342 23993 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.562062 23582 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b reported cstate change: config changed from index 9 to 10, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.562227 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 34a278486c8b447d8dc4c3241743de60 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.562513 24522 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.564169 24128 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.564868 23732 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:47.570672 24371 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a278486c8b447d8dc4c3241743de60 with delete_type TABLET_DATA_TOMBSTONED (TS 7e6e9293a4614dabb5ad89fe3f4d5d04 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:44666
I20240501 14:10:47.571035 29986 tablet_replica.cc:330] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: stopping tablet replica
I20240501 14:10:47.571240 29986 raft_consensus.cc:2229] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.571592 29986 raft_consensus.cc:2258] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.572360 29986 ts_tablet_manager.cc:1875] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.575945 29986 ts_tablet_manager.cc:1888] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 14:10:47.576063 29986 log.cc:1196] T 34a278486c8b447d8dc4c3241743de60 P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/wals/34a278486c8b447d8dc4c3241743de60
I20240501 14:10:47.576573 23574 catalog_manager.cc:4830] TS 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897): tablet 34a278486c8b447d8dc4c3241743de60 (table rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]) successfully deleted
I20240501 14:10:47.578454 24392 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e"
dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46076
I20240501 14:10:47.578600 24392 raft_consensus.cc:595] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:47.621201 24392 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e"
dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46084
I20240501 14:10:47.621313 24392 raft_consensus.cc:595] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:47.650688 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:47.650897 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:47.657156 24392 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e"
dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46096
I20240501 14:10:47.657284 24392 raft_consensus.cc:595] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:47.658829 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:47.660768 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:47.671474 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:47.672835 25921 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:47.675485 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:47.683827 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:47.684391 24371 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 14:10:47.684968 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:47.781391 23732 consensus_queue.cc:237] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } }
I20240501 14:10:47.782930 24257 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:47.782953 23862 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:47.782951 24128 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:47.783561 25808 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:47.783592 26071 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:47.783663 24392 raft_consensus.cc:1261] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 24d90b55560e45feb5fe4b9482317a4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:47.783950 29845 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:47.784138 29868 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:47.784298 29866 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:47.784438 29830 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:47.784657 29867 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:47.784780 29868 consensus_queue.cc:1035] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:47.786532 29866 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 24d90b55560e45feb5fe4b9482317a4c [term 1 LEADER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.787025 24392 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.787674 24257 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.787722 26071 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.790372 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d25ef07189ac43749efc549299dc89f8 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:47.790252 23583 catalog_manager.cc:5484] T d25ef07189ac43749efc549299dc89f8 P 6d832f3f723641778e60b3fd855ff98e reported cstate change: config changed from index 15 to 16, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New cstate: current_term: 1 leader_uuid: "24d90b55560e45feb5fe4b9482317a4c" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.790674 23862 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.790704 25808 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.791569 24128 raft_consensus.cc:2932] T d25ef07189ac43749efc549299dc89f8 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 40b69f74de5a447f86c1445f8e84850d (127.6.153.199) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } attrs { promote: false } } }
I20240501 14:10:47.796842 24504 tablet_service.cc:1514] Processing DeleteTablet for tablet d25ef07189ac43749efc549299dc89f8 with delete_type TABLET_DATA_TOMBSTONED (TS 40b69f74de5a447f86c1445f8e84850d not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:58450
I20240501 14:10:47.797384 29876 tablet_replica.cc:330] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: stopping tablet replica
I20240501 14:10:47.797530 29876 raft_consensus.cc:2229] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:47.797850 29876 raft_consensus.cc:2258] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:47.798444 29876 ts_tablet_manager.cc:1875] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:47.800114 29876 ts_tablet_manager.cc:1888] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.15
I20240501 14:10:47.800189 29876 log.cc:1196] T d25ef07189ac43749efc549299dc89f8 P 40b69f74de5a447f86c1445f8e84850d: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-6/wal/wals/d25ef07189ac43749efc549299dc89f8
I20240501 14:10:47.800586 23576 catalog_manager.cc:4830] TS 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305): tablet d25ef07189ac43749efc549299dc89f8 (table rebalance_test_table_1 [id=a3e1dc005de445cca21039f88ee3fcd8]) successfully deleted
I20240501 14:10:47.883775 24392 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e"
dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46114
I20240501 14:10:47.883891 24392 raft_consensus.cc:595] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:47.892716 24392 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e"
dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46122
I20240501 14:10:47.892836 24392 raft_consensus.cc:595] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Received request to transfer leadership
I20240501 14:10:47.934886 29864 raft_consensus.cc:979] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: : Instructing follower c958143102d04279ac25aa56214d0d4b to start an election
I20240501 14:10:47.935002 29864 raft_consensus.cc:1067] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Signalling peer c958143102d04279ac25aa56214d0d4b to start an election
I20240501 14:10:47.935552 25940 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e"
dest_uuid: "c958143102d04279ac25aa56214d0d4b"
from {username='slave'} at 127.6.153.198:33603
I20240501 14:10:47.935708 25940 raft_consensus.cc:482] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 14:10:47.935786 25940 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:47.936537 25940 raft_consensus.cc:504] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.937776 25940 leader_election.cc:288] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [CANDIDATE]: Term 2 election: Requested vote from peers 2502b9e8098b4bc5af7204835fce444b (127.6.153.194:34427), 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621), 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313), dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101), 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897), 40b69f74de5a447f86c1445f8e84850d (127.6.153.199:35305), 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:47.938071 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "c958143102d04279ac25aa56214d0d4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "24d90b55560e45feb5fe4b9482317a4c"
I20240501 14:10:47.938143 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "c958143102d04279ac25aa56214d0d4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "2502b9e8098b4bc5af7204835fce444b"
I20240501 14:10:47.938148 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "c958143102d04279ac25aa56214d0d4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04"
I20240501 14:10:47.938194 23732 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:47.938242 23862 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:47.938262 24392 raft_consensus.cc:3032] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 14:10:47.938302 24392 raft_consensus.cc:729] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7e6e9293a4614dabb5ad89fe3f4d5d04, State: Running, Role: LEADER
I20240501 14:10:47.938448 24392 consensus_queue.cc:260] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.938711 24392 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:47.939143 23862 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c958143102d04279ac25aa56214d0d4b in term 2.
I20240501 14:10:47.939708 24392 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c958143102d04279ac25aa56214d0d4b in term 2.
I20240501 14:10:47.939790 24257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "c958143102d04279ac25aa56214d0d4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "dbeb6a07fa7a44019ae00c169c414529"
I20240501 14:10:47.939904 24257 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:47.940794 24257 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c958143102d04279ac25aa56214d0d4b in term 2.
I20240501 14:10:47.941242 24128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "c958143102d04279ac25aa56214d0d4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "166d7170d98944e88de2eccda9d30545"
I20240501 14:10:47.941354 24128 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:47.942309 24128 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c958143102d04279ac25aa56214d0d4b in term 2.
I20240501 14:10:47.943303 25876 leader_election.cc:302] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 8 voters: 5 yes votes; 0 no votes. yes voters: 166d7170d98944e88de2eccda9d30545, 2502b9e8098b4bc5af7204835fce444b, 7e6e9293a4614dabb5ad89fe3f4d5d04, c958143102d04279ac25aa56214d0d4b, dbeb6a07fa7a44019ae00c169c414529; no voters:
I20240501 14:10:47.943375 23993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "c958143102d04279ac25aa56214d0d4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "8ebe0a098b0f4ed4861a5605717fecbd"
I20240501 14:10:47.943483 23993 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:47.943553 29121 raft_consensus.cc:2781] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 2 FOLLOWER]: Leader election won for term 2
I20240501 14:10:47.943728 29121 raft_consensus.cc:686] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 2 LEADER]: Becoming Leader. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Running, Role: LEADER
I20240501 14:10:47.943789 23732 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c958143102d04279ac25aa56214d0d4b in term 2.
I20240501 14:10:47.943995 29121 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:47.944032 24522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76fbefe89b8344e1a09101d404f32d9e" candidate_uuid: "c958143102d04279ac25aa56214d0d4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "40b69f74de5a447f86c1445f8e84850d"
I20240501 14:10:47.944145 24522 raft_consensus.cc:3037] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Advancing to term 2
I20240501 14:10:47.944448 23993 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c958143102d04279ac25aa56214d0d4b in term 2.
I20240501 14:10:47.945015 24522 raft_consensus.cc:2454] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c958143102d04279ac25aa56214d0d4b in term 2.
I20240501 14:10:47.946666 23581 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b reported cstate change: term changed from 1 to 2, leader changed from 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) to c958143102d04279ac25aa56214d0d4b (127.6.153.202). New cstate: current_term: 2 leader_uuid: "c958143102d04279ac25aa56214d0d4b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:48.343392 24257 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:48.343878 30104 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:48.362401 23862 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:48.362774 30104 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:48.426645 24128 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:48.426666 24522 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:48.427034 29121 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:48.427232 30117 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:48.427892 24392 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:48.428682 30104 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:48.430086 23732 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:48.432308 23993 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 14:10:48.430873 30120 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20240501 14:10:48.435209 30121 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 14:10:48.974834 25940 consensus_queue.cc:237] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 9, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:48.976329 24128 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:48.976423 24522 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:48.976509 24260 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:48.976723 23862 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:48.976735 30145 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:48.976961 30146 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:48.977116 30120 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:48.977231 23732 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:48.977329 30147 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:48.977497 23993 raft_consensus.cc:1261] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 2 FOLLOWER]: Refusing update from remote peer c958143102d04279ac25aa56214d0d4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 14:10:48.977658 30147 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:48.978278 30146 consensus_queue.cc:1035] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:48.979764 30148 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:48.980028 23862 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 2502b9e8098b4bc5af7204835fce444b [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:48.981333 23574 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 76fbefe89b8344e1a09101d404f32d9e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:48.981885 23732 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 24d90b55560e45feb5fe4b9482317a4c [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:48.981966 24260 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P dbeb6a07fa7a44019ae00c169c414529 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:48.982630 24128 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 166d7170d98944e88de2eccda9d30545 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:48.982748 23582 catalog_manager.cc:5484] T 76fbefe89b8344e1a09101d404f32d9e P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 9 to 11, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New cstate: current_term: 2 leader_uuid: "c958143102d04279ac25aa56214d0d4b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:48.984208 24522 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 40b69f74de5a447f86c1445f8e84850d [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:48.984879 23993 raft_consensus.cc:2932] T 76fbefe89b8344e1a09101d404f32d9e P 8ebe0a098b0f4ed4861a5605717fecbd [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:48.989358 24371 tablet_service.cc:1514] Processing DeleteTablet for tablet 76fbefe89b8344e1a09101d404f32d9e with delete_type TABLET_DATA_TOMBSTONED (TS 7e6e9293a4614dabb5ad89fe3f4d5d04 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:44666
I20240501 14:10:48.990177 30153 tablet_replica.cc:330] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: stopping tablet replica
I20240501 14:10:48.990327 30153 raft_consensus.cc:2229] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:48.990444 30153 raft_consensus.cc:2258] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:48.991149 30153 ts_tablet_manager.cc:1875] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:48.992939 30153 ts_tablet_manager.cc:1888] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.10
I20240501 14:10:48.993016 30153 log.cc:1196] T 76fbefe89b8344e1a09101d404f32d9e P 7e6e9293a4614dabb5ad89fe3f4d5d04: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-5/wal/wals/76fbefe89b8344e1a09101d404f32d9e
I20240501 14:10:48.993469 23574 catalog_manager.cc:4830] TS 7e6e9293a4614dabb5ad89fe3f4d5d04 (127.6.153.198:41897): tablet 76fbefe89b8344e1a09101d404f32d9e (table rebalance_test_table_0 [id=4f379bbc2480491993664b551317b221]) successfully deleted
I20240501 14:10:49.189457 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:49.192018 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:49.193987 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:49.195269 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.210960 24371 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.211050 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:49.211297 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.213258 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:49.219478 25921 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:49.253319 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:49.340685 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.341289 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:49.346755 24371 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.349400 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:49.350581 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:49.353152 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:49.355831 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.362854 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:49.372395 25921 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:49.375036 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:49.577112 23583 ts_manager.cc:292] Set tserver state for 24d90b55560e45feb5fe4b9482317a4c to MAINTENANCE_MODE
I20240501 14:10:49.777719 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.779654 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:49.781103 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:49.784559 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:49.792076 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.792470 24371 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.806474 25921 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:49.809056 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:49.816571 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:49.818414 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:49.929316 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:49.930233 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.933746 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:49.935379 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:49.937726 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:49.943699 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.945978 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:49.970312 24371 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:49.971417 25921 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:49.983515 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:50.113889 24108 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:50.114399 23710 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:50.114982 23844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:50.125532 25772 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:50.128280 24371 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:50.130136 26052 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 14:10:50.132611 23976 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 14:10:50.136498 24504 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 14:10:50.164016 24239 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 14:10:50.167198 25921 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 14:10:50.202418 24128 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:50.203904 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:50.204092 25940 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:50.204113 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:50.204427 23729 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:50.204564 23862 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:50.204605 30392 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:50.204742 23993 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:50.204771 29798 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:50.204921 30391 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:50.205143 30393 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:50.205360 30395 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:50.205551 29798 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:50.208096 30395 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:50.208540 23729 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:50.208624 23993 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:50.210063 25940 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:50.210403 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:50.211177 23589 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 13 to 14. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:50.212939 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:50.214865 23862 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } }
I20240501 14:10:50.218003 24128 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:50.220274 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:50.220374 23993 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:50.220738 30393 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:50.220773 23729 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
W20240501 14:10:50.221109 24061 consensus_peers.cc:485] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 -> Peer a308c0ed71f44f57967c97239d1198f7 (127.6.153.201:36409): Couldn't send request to peer a308c0ed71f44f57967c97239d1198f7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:50.221230 30392 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:50.221554 25940 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:50.221541 30392 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:50.221822 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:50.222373 30391 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:50.222682 23862 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 14:10:50.222708 30391 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:50.223599 29798 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 14:10:50.224282 30393 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:50.224645 23729 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:50.226576 23576 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet de25122a01e845d2b35e4a1023789d3e with cas_config_opid_index 14: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:50.226444 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:50.226833 23862 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:50.227007 25940 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:50.227244 23993 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:50.228940 23581 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c reported cstate change: config changed from index 14 to 15, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:50.230357 24392 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } } }
I20240501 14:10:50.236402 23993 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:50.238134 24522 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.238363 23729 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.238381 25808 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.238484 29981 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.238651 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.238790 29981 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.239445 24128 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.239445 26071 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.239764 29981 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.240036 29981 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.240265 29981 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.240572 30414 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.242514 30414 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:50.243163 25808 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:50.245066 24522 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:50.245306 23729 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:50.245507 26071 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:50.246366 23589 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 reported cstate change: config changed from index 10 to 11. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:50.247574 24392 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:50.252038 23993 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:50.252415 24128 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:50.254163 24392 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.254333 24522 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.254556 30414 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.254640 25808 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.254740 23729 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.255020 30414 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.255390 30402 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.255555 30414 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.255944 24128 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.256940 30430 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.258829 26071 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.258858 30402 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:50.259263 30414 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.260162 26071 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:50.260380 23577 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with cas_config_opid_index 11: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:50.260640 24127 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:50.261122 23729 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
W20240501 14:10:50.261319 23929 consensus_peers.cc:485] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd -> Peer c958143102d04279ac25aa56214d0d4b (127.6.153.202:34929): Couldn't send request to peer c958143102d04279ac25aa56214d0d4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fc754b4d0f144c4ba0bd9a1b77f7bb4. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:50.261646 23586 catalog_manager.cc:5484] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 11 to 12, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New cstate: current_term: 1 leader_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 14:10:50.262961 25808 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:50.263736 24522 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:50.265354 24391 raft_consensus.cc:2932] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER c958143102d04279ac25aa56214d0d4b (127.6.153.202) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } } }
I20240501 14:10:50.271569 24260 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } }
I20240501 14:10:50.273191 23993 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.273195 24128 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.273197 23729 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.273317 23862 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.273630 29879 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.273734 24522 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.273963 29879 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.274125 30242 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.274434 29879 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.274798 25940 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 14:10:50.275199 30242 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.275770 30242 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 14:10:50.277357 30435 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:50.277793 24128 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:50.278122 25940 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:50.279395 23862 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:50.279525 24522 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:50.280185 23729 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:50.282191 23589 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b reported cstate change: config changed from index 10 to 11. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:50.283560 23993 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } }
I20240501 14:10:50.287385 24260 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:50.289093 24128 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.289445 24522 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.289449 30433 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.289572 23729 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.289626 23862 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.290055 25940 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 14:10:50.290035 30435 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.290182 30433 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.290447 23993 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
W20240501 14:10:50.290526 24193 consensus_peers.cc:485] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 -> Peer 6d832f3f723641778e60b3fd855ff98e (127.6.153.200:37319): Couldn't send request to peer 6d832f3f723641778e60b3fd855ff98e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34a278486c8b447d8dc4c3241743de60. This is attempt 1: this message will repeat every 5th retry.
I20240501 14:10:50.290609 30434 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.290789 30433 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.291841 30433 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 14:10:50.292992 30433 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:50.293936 24128 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:50.294008 25940 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:50.294966 23732 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:50.296434 23993 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:50.296586 23862 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:50.298199 23581 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 reported cstate change: config changed from index 11 to 12, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:50.298386 23575 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 34a278486c8b447d8dc4c3241743de60 with cas_config_opid_index 11: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 14:10:50.298848 24522 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } } }
I20240501 14:10:50.682264 30476 ts_tablet_manager.cc:902] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Initiating tablet copy from peer 166d7170d98944e88de2eccda9d30545 (127.6.153.196:46669)
I20240501 14:10:50.682814 30476 tablet_copy_client.cc:308] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.196:46669
I20240501 14:10:50.683301 24148 tablet_copy_service.cc:140] P 166d7170d98944e88de2eccda9d30545: Received BeginTabletCopySession request for tablet de25122a01e845d2b35e4a1023789d3e from peer a308c0ed71f44f57967c97239d1198f7 ({username='slave'} at 127.6.153.201:41233)
I20240501 14:10:50.683430 24148 tablet_copy_service.cc:161] P 166d7170d98944e88de2eccda9d30545: Beginning new tablet copy session on tablet de25122a01e845d2b35e4a1023789d3e from peer a308c0ed71f44f57967c97239d1198f7 at {username='slave'} at 127.6.153.201:41233: session id = a308c0ed71f44f57967c97239d1198f7-de25122a01e845d2b35e4a1023789d3e
I20240501 14:10:50.684126 24148 tablet_copy_source_session.cc:205] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:50.684741 30476 ts_tablet_manager.cc:1875] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 14:10:50.686446 30476 ts_tablet_manager.cc:1888] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.12
I20240501 14:10:50.686607 30476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de25122a01e845d2b35e4a1023789d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:50.687572 30476 tablet_copy_client.cc:783] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:50.687687 30476 tablet_copy_client.cc:647] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:50.688737 30476 tablet_copy_client.cc:523] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:50.689572 30476 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Bootstrap starting.
I20240501 14:10:50.711002 30476 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:50.711239 30476 tablet_bootstrap.cc:492] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Bootstrap complete.
I20240501 14:10:50.711431 30476 ts_tablet_manager.cc:1367] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Time spent bootstrapping tablet: real 0.022s user 0.018s sys 0.004s
I20240501 14:10:50.711791 30476 raft_consensus.cc:348] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:50.711922 30476 raft_consensus.cc:729] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a308c0ed71f44f57967c97239d1198f7, State: Initialized, Role: LEARNER
I20240501 14:10:50.712033 30476 consensus_queue.cc:260] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: NON_VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: true } }
I20240501 14:10:50.712301 30476 ts_tablet_manager.cc:1398] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 14:10:50.712774 24148 tablet_copy_service.cc:342] P 166d7170d98944e88de2eccda9d30545: Request end of tablet copy session a308c0ed71f44f57967c97239d1198f7-de25122a01e845d2b35e4a1023789d3e received from {username='slave'} at 127.6.153.201:41233
I20240501 14:10:50.712847 24148 tablet_copy_service.cc:433] P 166d7170d98944e88de2eccda9d30545: ending tablet copy session a308c0ed71f44f57967c97239d1198f7-de25122a01e845d2b35e4a1023789d3e on tablet de25122a01e845d2b35e4a1023789d3e with peer a308c0ed71f44f57967c97239d1198f7
I20240501 14:10:50.759438 30497 ts_tablet_manager.cc:902] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: Initiating tablet copy from peer dbeb6a07fa7a44019ae00c169c414529 (127.6.153.197:38101)
I20240501 14:10:50.759842 30497 tablet_copy_client.cc:308] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.197:38101
I20240501 14:10:50.760270 24280 tablet_copy_service.cc:140] P dbeb6a07fa7a44019ae00c169c414529: Received BeginTabletCopySession request for tablet 34a278486c8b447d8dc4c3241743de60 from peer 6d832f3f723641778e60b3fd855ff98e ({username='slave'} at 127.6.153.200:47351)
I20240501 14:10:50.760355 24280 tablet_copy_service.cc:161] P dbeb6a07fa7a44019ae00c169c414529: Beginning new tablet copy session on tablet 34a278486c8b447d8dc4c3241743de60 from peer 6d832f3f723641778e60b3fd855ff98e at {username='slave'} at 127.6.153.200:47351: session id = 6d832f3f723641778e60b3fd855ff98e-34a278486c8b447d8dc4c3241743de60
I20240501 14:10:50.761119 24280 tablet_copy_source_session.cc:205] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:50.761909 30497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34a278486c8b447d8dc4c3241743de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:50.763693 30497 tablet_copy_client.cc:783] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:50.763901 30497 tablet_copy_client.cc:647] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:50.764947 30497 tablet_copy_client.cc:523] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:50.765930 30497 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap starting.
I20240501 14:10:50.785918 30497 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:50.786166 30497 tablet_bootstrap.cc:492] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: Bootstrap complete.
I20240501 14:10:50.786332 30497 ts_tablet_manager.cc:1367] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20240501 14:10:50.786700 30497 raft_consensus.cc:348] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:50.786808 30497 raft_consensus.cc:729] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6d832f3f723641778e60b3fd855ff98e, State: Initialized, Role: LEARNER
I20240501 14:10:50.786896 30497 consensus_queue.cc:260] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: NON_VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: true } }
I20240501 14:10:50.787263 30497 ts_tablet_manager.cc:1398] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:10:50.787768 24280 tablet_copy_service.cc:342] P dbeb6a07fa7a44019ae00c169c414529: Request end of tablet copy session 6d832f3f723641778e60b3fd855ff98e-34a278486c8b447d8dc4c3241743de60 received from {username='slave'} at 127.6.153.200:47351
I20240501 14:10:50.787849 24280 tablet_copy_service.cc:433] P dbeb6a07fa7a44019ae00c169c414529: ending tablet copy session 6d832f3f723641778e60b3fd855ff98e-34a278486c8b447d8dc4c3241743de60 on tablet 34a278486c8b447d8dc4c3241743de60 with peer 6d832f3f723641778e60b3fd855ff98e
I20240501 14:10:50.822288 30505 ts_tablet_manager.cc:902] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: Initiating tablet copy from peer 8ebe0a098b0f4ed4861a5605717fecbd (127.6.153.195:44313)
I20240501 14:10:50.822765 30505 tablet_copy_client.cc:308] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.153.195:44313
I20240501 14:10:50.823155 24016 tablet_copy_service.cc:140] P 8ebe0a098b0f4ed4861a5605717fecbd: Received BeginTabletCopySession request for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 from peer c958143102d04279ac25aa56214d0d4b ({username='slave'} at 127.6.153.202:45507)
I20240501 14:10:50.823243 24016 tablet_copy_service.cc:161] P 8ebe0a098b0f4ed4861a5605717fecbd: Beginning new tablet copy session on tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 from peer c958143102d04279ac25aa56214d0d4b at {username='slave'} at 127.6.153.202:45507: session id = c958143102d04279ac25aa56214d0d4b-0fc754b4d0f144c4ba0bd9a1b77f7bb4
I20240501 14:10:50.824119 24016 tablet_copy_source_session.cc:205] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 14:10:50.824927 30505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 14:10:50.826505 30505 tablet_copy_client.cc:783] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 0 data blocks...
I20240501 14:10:50.826637 30505 tablet_copy_client.cc:647] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: tablet copy: Starting download of 1 WAL segments...
I20240501 14:10:50.827567 30505 tablet_copy_client.cc:523] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 14:10:50.828625 30505 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: Bootstrap starting.
I20240501 14:10:50.846901 30505 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 14:10:50.847113 30505 tablet_bootstrap.cc:492] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: Bootstrap complete.
I20240501 14:10:50.847270 30505 ts_tablet_manager.cc:1367] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.000s
I20240501 14:10:50.847602 30505 raft_consensus.cc:348] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:50.847707 30505 raft_consensus.cc:729] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c958143102d04279ac25aa56214d0d4b, State: Initialized, Role: LEARNER
I20240501 14:10:50.847791 30505 consensus_queue.cc:260] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: NON_VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: true } }
I20240501 14:10:50.848011 30505 ts_tablet_manager.cc:1398] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 14:10:50.848593 24016 tablet_copy_service.cc:342] P 8ebe0a098b0f4ed4861a5605717fecbd: Request end of tablet copy session c958143102d04279ac25aa56214d0d4b-0fc754b4d0f144c4ba0bd9a1b77f7bb4 received from {username='slave'} at 127.6.153.202:45507
I20240501 14:10:50.848671 24016 tablet_copy_service.cc:433] P 8ebe0a098b0f4ed4861a5605717fecbd: ending tablet copy session c958143102d04279ac25aa56214d0d4b-0fc754b4d0f144c4ba0bd9a1b77f7bb4 on tablet 0fc754b4d0f144c4ba0bd9a1b77f7bb4 with peer c958143102d04279ac25aa56214d0d4b
I20240501 14:10:51.201064 25808 raft_consensus.cc:1203] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.14->[1.15-1.15] Dedup: 1.15->[]
I20240501 14:10:51.328626 26072 raft_consensus.cc:1203] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 14:10:51.446167 25940 raft_consensus.cc:1203] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 14:10:51.612859 30394 raft_consensus.cc:1050] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545: attempting to promote NON_VOTER a308c0ed71f44f57967c97239d1198f7 to VOTER
I20240501 14:10:51.613376 30394 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:51.615142 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:51.615312 23993 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:51.615474 23862 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:51.615748 25940 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:51.615876 24391 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:51.616240 25793 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 LEARNER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:51.616271 30568 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:51.616458 30543 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:51.616576 30571 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:51.616982 30568 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:51.617223 23729 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 14:10:51.617934 30542 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:51.618258 30568 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:51.619372 30543 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 14:10:51.621255 30569 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.16: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.621919 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.623250 25793 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.623261 23862 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.625883 25940 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.626132 24391 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.626240 23586 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 reported cstate change: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.628141 23993 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.630178 23729 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, a308c0ed71f44f57967c97239d1198f7 (127.6.153.201) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.633338 24128 consensus_queue.cc:237] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 1.16, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } }
I20240501 14:10:51.634759 23993 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 14:10:51.635293 24392 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 14:10:51.635552 24260 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 14:10:51.635555 25941 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 14:10:51.635668 30568 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 14:10:51.635887 30543 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 14:10:51.636014 30542 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 14:10:51.636160 23862 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 14:10:51.636240 25793 raft_consensus.cc:1261] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 166d7170d98944e88de2eccda9d30545: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 14:10:51.636581 30542 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 14:10:51.637746 30571 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 14:10:51.639511 30568 consensus_queue.cc:1035] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 14:10:51.639925 23993 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.640167 24391 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.641391 30569 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 166d7170d98944e88de2eccda9d30545 [term 1 LEADER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.642246 23582 catalog_manager.cc:5484] T de25122a01e845d2b35e4a1023789d3e P 8ebe0a098b0f4ed4861a5605717fecbd reported cstate change: config changed from index 16 to 17, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "166d7170d98944e88de2eccda9d30545" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.643986 24260 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P dbeb6a07fa7a44019ae00c169c414529 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.645658 25941 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.653910 23710 tablet_service.cc:1514] Processing DeleteTablet for tablet de25122a01e845d2b35e4a1023789d3e with delete_type TABLET_DATA_TOMBSTONED (TS 24d90b55560e45feb5fe4b9482317a4c not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:55572
I20240501 14:10:51.655413 30594 tablet_replica.cc:330] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: stopping tablet replica
I20240501 14:10:51.655360 25793 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.655771 30594 raft_consensus.cc:2229] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:51.656190 30594 raft_consensus.cc:2258] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:51.657285 23576 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet de25122a01e845d2b35e4a1023789d3e with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:51.658576 30594 ts_tablet_manager.cc:1875] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 14:10:51.659359 23862 raft_consensus.cc:2932] T de25122a01e845d2b35e4a1023789d3e P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } }
I20240501 14:10:51.660892 30594 ts_tablet_manager.cc:1888] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.16
I20240501 14:10:51.660979 30594 log.cc:1196] T de25122a01e845d2b35e4a1023789d3e P 24d90b55560e45feb5fe4b9482317a4c: Deleting WAL directory at /tmp/dist-test-taskxMccU0/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714572527634482-6759-0/raft_consensus-itest-cluster/ts-0/wal/wals/de25122a01e845d2b35e4a1023789d3e
I20240501 14:10:51.661556 23575 catalog_manager.cc:4830] TS 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193:35621): tablet de25122a01e845d2b35e4a1023789d3e (table rebalance_test_table_6 [id=5a5ab14c21db4ed3abc85ba0d4275a91]) successfully deleted
I20240501 14:10:51.806742 30544 raft_consensus.cc:1050] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529: attempting to promote NON_VOTER 6d832f3f723641778e60b3fd855ff98e to VOTER
I20240501 14:10:51.807219 30544 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:51.809011 26072 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e [term 1 LEARNER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.809365 24128 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.809661 25941 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.809715 23729 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.809715 23993 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.809769 24522 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.810045 30597 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.810357 30598 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.810441 23862 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.810514 30544 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.810684 30504 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.810833 30448 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.810951 30601 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.811110 30600 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.814599 30598 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.815042 24128 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.815263 23729 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.816583 25941 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.817417 23588 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:51.817566 26072 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.818250 24522 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.820469 23862 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.821439 23993 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 6d832f3f723641778e60b3fd855ff98e (127.6.153.200) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.826138 24260 consensus_queue.cc:237] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } }
I20240501 14:10:51.826537 30413 raft_consensus.cc:1050] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd: attempting to promote NON_VOTER c958143102d04279ac25aa56214d0d4b to VOTER
I20240501 14:10:51.827003 30413 consensus_queue.cc:237] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } } peers { permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } }
I20240501 14:10:51.827517 26072 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:51.828306 30448 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:51.828394 23862 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 2502b9e8098b4bc5af7204835fce444b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:51.828771 30599 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:51.828913 24128 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:51.829173 26071 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.829360 25793 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P a308c0ed71f44f57967c97239d1198f7 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.829437 24391 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 7e6e9293a4614dabb5ad89fe3f4d5d04 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.829581 30413 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.829749 30448 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:51.830083 25941 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P c958143102d04279ac25aa56214d0d4b [term 1 LEARNER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.830119 25940 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:51.830539 23993 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 8ebe0a098b0f4ed4861a5605717fecbd [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:51.830603 23729 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.830782 24524 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.831064 24127 raft_consensus.cc:1261] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Refusing update from remote peer 8ebe0a098b0f4ed4861a5605717fecbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 14:10:51.831140 30611 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.831431 30448 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:51.831609 30416 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.831934 30416 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a308c0ed71f44f57967c97239d1198f7" member_type: VOTER last_known_addr { host: "127.6.153.201" port: 36409 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.832048 30448 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:51.832226 30416 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e6e9293a4614dabb5ad89fe3f4d5d04" member_type: VOTER last_known_addr { host: "127.6.153.198" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.832494 30416 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d90b55560e45feb5fe4b9482317a4c" member_type: VOTER last_known_addr { host: "127.6.153.193" port: 35621 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.832803 30561 consensus_queue.cc:1035] T 0fc754b4d0f144c4ba0bd9a1b77f7bb4 P 8ebe0a098b0f4ed4861a5605717fecbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 14:10:51.833091 30448 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.834645 25941 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P c958143102d04279ac25aa56214d0d4b [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.834951 24522 raft_consensus.cc:1261] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Refusing update from remote peer dbeb6a07fa7a44019ae00c169c414529: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 14:10:51.835059 23575 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 34a278486c8b447d8dc4c3241743de60 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 14:10:51.835412 30544 consensus_queue.cc:1035] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 14:10:51.835309 26072 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 6d832f3f723641778e60b3fd855ff98e [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.836323 23581 catalog_manager.cc:5484] T 34a278486c8b447d8dc4c3241743de60 P dbeb6a07fa7a44019ae00c169c414529 reported cstate change: config changed from index 13 to 14, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New cstate: current_term: 1 leader_uuid: "dbeb6a07fa7a44019ae00c169c414529" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 14:10:51.837038 24127 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 166d7170d98944e88de2eccda9d30545 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "6d832f3f723641778e60b3fd855ff98e" member_type: VOTER last_known_addr { host: "127.6.153.200" port: 37319 } attrs { promote: false } } }
I20240501 14:10:51.839232 23710 tablet_service.cc:1514] Processing DeleteTablet for tablet 34a278486c8b447d8dc4c3241743de60 with delete_type TABLET_DATA_TOMBSTONED (TS 24d90b55560e45feb5fe4b9482317a4c not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:55572
I20240501 14:10:51.839455 30594 tablet_replica.cc:330] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c: stopping tablet replica
I20240501 14:10:51.839577 30594 raft_consensus.cc:2229] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 14:10:51.839859 30594 raft_consensus.cc:2258] T 34a278486c8b447d8dc4c3241743de60 P 24d90b55560e45feb5fe4b9482317a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 14:10:51.840111 24522 raft_consensus.cc:2932] T 34a278486c8b447d8dc4c3241743de60 P 40b69f74de5a447f86c1445f8e84850d [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER 24d90b55560e45feb5fe4b9482317a4c (127.6.153.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2502b9e8098b4bc5af7204835fce444b" member_type: VOTER last_known_addr { host: "127.6.153.194" port: 34427 } } peers { permanent_uuid: "8ebe0a098b0f4ed4861a5605717fecbd" member_type: VOTER last_known_addr { host: "127.6.153.195" port: 44313 } } peers { permanent_uuid: "dbeb6a07fa7a44019ae00c169c414529" member_type: VOTER last_known_addr { host: "127.6.153.197" port: 38101 } } peers { permanent_uuid: "40b69f74de5a447f86c1445f8e84850d" member_type: VOTER last_known_addr { host: "127.6.153.199" port: 35305 } } peers { permanent_uuid: "c958143102d04279ac25aa56214d0d4b" member_type: VOTER last_known_addr { host: "127.6.153.202" port: 34929 } attrs { promote: false } } peers { permanent_uuid: "166d7170d98944e88de2eccda9d30545" member_type: VOTER last_known_addr { host: "127.6.153.196" port: 46669 } attrs { promote: false } } peers { permanent_uuid: "